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that consists of both written and perforaance tests in each of five 
areas: chairside assistino, patient, education, diagnostic aids, 
therapeutics, and laboratory procedures. The content cutlines of - the 
tests- are presented. The f craation of the conaittees and the 
processes of test planning and test developaent are described* 
Checklists were developed for the observers to use in scoring the 
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a three-stage process. Inter-observer teliabilities were computed for 
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ra«ed adeguate with those rated inadeguate by the observers. Various 
methods were used to select recoiaended pass/ fail cutcff scores. • 
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are presented. <Author/CTM) H 
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ABSTRACT 



This report describes the development of a criterion-referenced 
Proficiency Examination for dental auxiliaries. It consists of written 
multiple-choice tests and performance (practical) tests ia each of five 
areas: chairside assisting, patient education, diagnostic aids, therapeu- 
tics, and laboratory procedures. The written tests emphasize -knowledge of 
correct procedures needed on the job. The performance tests consist of 
selected tasks to be performed under standardized conditions ~ some on 
real patients, others on patient-actors or dental manikins. They include 
evaluations of both process (procedure) and product. Examinee performance 
is recorded by observers using checklists. These performance checklists 
are highly specific, to minimize differences between observers. Scoring 
forcnuLas, the same for all examinees, translate the recorded observations 
into numerical scores. 

The tests were developed by a ten-person Working Committee of dentists, 
dental assistants, and .dental hygienists, working closing with ETS staff, 
under the guidance of an Advisory Committee of persons nominated by associa- 
tions in the dental professions. The projjBCt included~^two tryout adminis- 
trations of the examination: a small-scale "pretest" (24 examinees) and a 
Inrp.er-scalo "pilot test" (115 examinees). Each was preceded by a two-day 
training, workshop for the performance test observers. The tests were revised 
extensively on the basis of the pretest; minor revisions were made after the 
pilot test. Score distributions and reliability statistics were computed 
from the pilot test data. 

/The report concludes with recommendations concerning the use of the 
examinat ion. 



INTRODUCTION 

This report descries the development of a prof iciency - exai^ination 
for. dental auxiliarieis — i.e., dental assir.tants and dental bygienlsts. ' 
A proficiency examination is an exafhlnation that is intended to reflect 
a person's ability to do a job, (or a. group of jobs, or a portion of a job)* 
A proficiency examination is not tlfe same thing as an academic equivalency 
examination: ^'Occupational proficiency examinations are intended to deter- 
mine whether the individual meets job knowledge requirements, which are 
not necessarily the same as the knowledge required to obtain a degree ia the 
field* These examinat ions .may also include a determination of whether the 
individual possesses the necessary skills to perform adequately*"* A 
proficiency exanination tests kndWledge and skills directly involved in the 

performance of the job and tests them in the context of practical applica- 

« 

tions on the job* In the context of the health professions, a proficiency 
examination has been defined as "a criterion-i^f erenced appraisal of the 
individual's possession of the'perf ormance knowledge and skill competencies 
required to execute the role responsibilities of the given practitioner 
generic posit ion . A proficiency examination is criterion-referenced in 
the sense that ^t yields a score or set of scores that are to be interpreted 
in terns ol an 4bsolute standard of proficiency, rather than in terms of the 
examinee's relative standing in a groups 

* M* Y* Pennell and D* B* Hoover, "Policie's for the development of 
credentraling mechanisms for health personnel," Operation MEDIHC , 
February, 1972, Vol. 2, no. 3. (New York: National Health Council) 

Margaret A* Wilson, "'An- introduction to the proficiency examination,*' 
American Journal of Occupational Therapy , March, 1977, Vol* 31, no. 3, 
pp. 162-168, 



The Proficiency Exandnation for Ddntal Auxiliaries consists both 
written tests and performance .(practical ) tests in each of five competency 
areas: chairside assisting, laboratory proc^clures , diagnostic aids.,-. patient 
education, and therapei^tics. The examination was designed to be a complete ' 
examination covering th^- range of functions nornally performed by dental 
auxiliaries. However,, to allow greater flexibility in its use, the exanina- 
tion is in "modular" format. That is, the written test and the performance 
test in each competency area constitute a separate "module" that can bo 
administered separately from the other tests. The written and performan^ 
components of each- module were designed to be used together. The written, 
component alone does not test the important physical skills and communication 
skills necessary to perform the tasks in the competenpy area. The perfor- . 
mance component aJ,one tests too s.mall a sample of skills and knowledge. 
But together, the written and performance components provide an effective 
tost of the knowledge and skills involved in performing the tasks in each 

I 

competency area. * 

The Dental Auxiliary Examination Project began with a.recfuest for pro- 
posals, issued by the Department of Health, Education, and^ Welfare, Bureau of 
Health Resources Development, Division of Dentistry (RFP No. HRA 231-7E-0579) 
The purpose of the project, as stated in the request for proposals, was "to 

develop a proficiency examination to appraise the entry-level knowledge and 

\ 

skill competencies of dental auxiliaries (dental hygienists and dental assis- 
tants) for use in the credentialing process." The legislative authorization 
of the project was stated as follows: 

4ft 

In recent years the Federal Government has assumed a significant 
role in the developmpnt of proficiency testing mechanisms through 
two specific authorities. The Health Trailing Improvement Act of 
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19.70 (P,L* 91-519) authorized contracts for "developing, demonstra- 
ting, or evaluating techniques for appropriate recognition (includ- 
ing equivalency and proficiency testing mechanisms) of previously 
acquired training or experience* • and the Social Security 
Amendments of 1972 (P.L. 92-603, Sec, 1123), which authorize the 
development of proficiency examinations until 1977 for personnel 
who do not meet formal educational, professional membership require-* 
ments, or other specific criteria established to determine quali- 
fications of health care personnel for reimbursement of services 
under Titles XVIII and XIX of the Act, The Division of Associated 
Health Professions (HRA) and the Bureau of Quality Assurance (HSA) 
have been carrying out the development of the examinations as 
authorized, and through a memorandum of agreement, have coordi- 
nated their activities to optimize their effort and avoid dupli- 
cation. It is intended that the instruments will be utilized by 
the professions to satisfy their credentialing program, bj '.-tate 
credentialing agencies, merit systems. Civil Service agencies, 
and other entities for purposes of assessing practitioner compe- 
tency. Further, it is <pected that proficiency tests will be 
based upon national performance standards of competency for cre- 
dentialing all health care personnel. This long-range goal is 
aimed at providing opportunities for geographic and career 
nobility for health personnel; establish a means of recognizing 
previously acqurlred training or experience; and providing access 
to health care by competent personnel* 



A contract to develop the examination was awarded to Educational 
Testing Service (LIS) in Octc r of 1977. ETS had previously worked with 
prol fssionn I organizations to develop credentialing examinations in several 
other health professions, including podiatry, obstetrics and gynecology, 
pharmacy, nursing, respiratory therapy, and radiologic technology* ETS had. 
also developed a series of college equivalency examinations in dental auxiliary 
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Description of the Examination 

The written tests 

The examination Includes a written multiple-choice test for each of the 
five competency areas. Although these tests wore designed to be administered 
at a single 3 1/2-hour session, they are separately timed and separately 
scored, and are not in any way dependent on each other* The tests are made 
up of four-choice questions. Three of the written tests (diagnostic aids, 
patient education, and dental therapeutics) exist in two versions: a shorter 
version intended for dental assistants and a longer version intended for 
dental hyglenlsts* The longer version Includes all of th^ questions in the 
shorter version, plus several additional questions. The reason for thu 
existence of two versions of the written tests in these areas is that each 
of the written tests covers several tasks, and some of .the tasks regularly 
performed by dental hyglenlsts are not performed by dental assistants (in most 
stages). The existence of the two versions doe^ not imply a different s^tand- 
ard of proficiency for dental assistants than for dental hygienists. On' the 
. ^ contrary, the development of the Proficiency Examination was based on the 

premise fchat the level of proficiigncv required of a pers9n who performs a task 

y 

should be the same regardless of whether that person is a dental ass ist ant or 
a dental hygienist * 

The content of the written tests concentrates heavily on knowledge of 
correct procedures. The emphasis is on practical knowledge — knowing what to 
do in a given situation on the job. The specific content of the written tests 
is summarized in the following paragraphs and in Table 1. 

Chairsld e assisting . This test includes questions on preparation and 
maintenance of the operafing area (including disinfection and aspiration), 

13 
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Table 1. The wriccen tests s Humber of questions on each topic < 



Chairside assisting 

Preparation and maintenance of the 
operating area (disinfection^ preparation 
of patient) 

Efficient utilization of dental auxiliaries 
(aspiration, clonrlng the field, delivering 
and rotrlevlng instruments) 

Restorative materials 

Impression materials 

Medical emergencies 



Patient education 



Oral hygitrne needs asst^ssmunt 

Or.U hyglonti techniques (including dietary 
an.i lysis and counseling) 

Evaluation of patients* progress 



Diagnostic aids '* 

Vital signs (blood pressure, pulse rate) 
Rad iology 

Modiral and dental history 

Pr--> I irnlnary oral t-xanlnation (Ir.cluding 
periodontal probing and ch.irting) 

TaV int; d la.?;nos t ic impress ipns 



Total 

Assistants' version 

13 
12 



14 

15 

16 
12 

JL 
64 



Total 



Total 



J, 
26 



10 
24 
7 
3 

_5 
49 



Hygienists' version 

17 
20 

_3 
40 



10 
26 
8 
9 

_7 
60 



'A,K't i, .-lA ICS 

Aj;»I: .iLion ot topiciil ,i/,i'nt s 

P.) 1 i in/ tcftri and res tor at ions 

P I iv on.-nt antj removal of t iswuo dresbJn^s 

Su' :ri' rfr. >v.il 

K. a I of calculus iJ»*poslts ( f nc 1 uJ i n^^^ 
r.).>t pi iMin/j anv! mar^;inal ovt-ruati^ 

Pr^-pariHAi stuJy castb 

\ abr icat ix^^ acr vl io tonpornry crovti rcstorac tons 
Fabricating; ^- us tarn acryli«: impression trays 
CU-arUn^ arid pollshln*i reru>vable appliances 
litrr i I i/at i'jn an. I disinfection of instruments 



Assistants' version 

4 

3 
6 
2 
0 



Total 



15 



Total 



6 

2 

6 

4 

10 
28 



Kygienists* version 
4 

13 

6 

2 
11 

36 
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delivering and retrieving instruments, preparinif^nd delivering restorative 
materials, preparing and delivering impression materials, and dealing with 
medical emergencies. This test is the same for assistants as for hygienists • 
It consists of 64 questions, and the time limit is 60 minutes. 

Patient education . This test exists in two versions, one for dental 
assistants and one for dental hygienists. Both versions include questions 
on oral hygiene needs assessment, oral hygiene techniques, dietary analysis, 
and evaluation of the patient's progress. The dental assistants' version of 
the test consists of 26 questions and has a time limit of 25 minutes; the 
dental hygienists' version consists of 40 questions . and has a time limit of 
33 minutes. 

Diagnost ic aids . This test also exists in two versions. The dental 
assistants' version includes questions on taking a meclical and dental history, 
taking vital signs, taking diagnostic impressions, and dental radiography. It 
consists of 49 questions and has a time limit of 45 minutes. The dental 
hygienists' version includes questions on the same topics as the assistants' 
version and also on performing a preliminary oral examination (including 
periodontal probing and charting). It consists of 60 questions and has a time 
limit of 55 minutes. 

Therapeutics. This test also exists in two versions. The dental 
assistants' version includes questions on the application of topical agents, 
polishing restorations, removal of marginal overhang, placement and removal 
of tissue dressings, and suture removal. It consists of fifteen questions and 
has a time limit of fifteen rainvites. The dental hygienists' version includes " 
questions on the same topics as the assistants' version and also on polishing 
teeth and the removal of calculus deposits (including root planing). It con- 
sists of 36 questions and has a time limit of 35 minutes. 




Laboratory orocedurea. This test includes questions on preparing 
stujiy casts, mnicing acrylic temporary crown restorations, making custom 
acrylic trays, cleaning and polishing removable appliances, and disinfect- 
ing and sterilizing instruments. The test is the same for assistants as for 
hygienists. It consists of 28 questions ^nd has a time limit of 25 minutes. 

The pgyformanc c tests 

A perfornancc test is one in which the examines (the person taking the 
tost) actually performs the tasks for which he/she is being tested. The 
critical problem in most performance tests is subjectivity — the fact that 
an ox.-mince's score may depend on the individual standards of the person who • 
adninistors and scores the test. In any test that calls for the examinee to 
demonstrate a procedure, the examinee's performance must be translated into 
a number or some other form that can serve as the basis for' a decision. The 
scoring process inevitably involves some degree of human judgment. The 
judgmental pare of the scoring process cannot he removed, but it can be 
.stniidardized, by the use of a technique called analytic scoring . This 
technique requires the test developers to analyze each task into a series of 
highly specific actions. Each action can then be observed and recorded .as 
being done or not done, with little chance of disagreement between observers. 
Thoso sptciflc actions are listed on a performance checklis t, which the 
observer uses to record the examinee's performance. The information on the 
checklist Is converted to a numerical score by means of a scoring formula 
that is applied uniformly to all examinees. 

Performance tests can test either process or product — or both. Obviously, 
if the task yields an observable product, it^ makes sense to evaluate the pro- 
duct, but it may be important to evaluate the process also. An incorrect 
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technique may produce an Inadequate product only part of the time, cr it may 
produce an adequate product but present an unnecessary danger to the patient. 
If the test evaluates only the product, it will axi^ard full credit to an 
examinee who uses such an incorrect technique but happens to , produce an 
adequate result on the occasion of the test. For this reason, most of the 
performance tests m the Proficiency Examination for Dental Auxiliaries 
include evaluations of both process and product. 

One of the more difficult questions of performance testing is the 

':> 

question of whic;- aspects of the task to standardize. This question applies 
to equipment, materials, procedures, and the behavior of other persons the 
auxiliary must work with. In general, the performance tests in this examina- 
tion do not specify a particular operatory layout. They do not require the 
standardization of i.arge equipment (e.g., x-ray units). They do require a 
particular assortment of small equipment (e.g., hand instruments) and dental 
materials, specified by type ]>ut not by manufacturer. Some aspects of the 
procedures are standardized (e.g., the number of films in the x-ray series); 

o 

others are not (e.g., the- charting system in charting from dictation). Where 
the auxiliary must work closely with another person who participates^ actively 
in the tr^-^k (e.g., the dentist), that person's behavior has been standardized. 

The examinee- is told in advance which specific tasks will be tested in 
the performance tests and whether each task will be performed on a manikin 
or on an actual patient. If there is more than one acceptable procedure for 
doing a task, but a particular procedure is specified for the test, the 
examinee is told in advance which procedure is. to be used. Some of the 
performance tests have time limits, and the examinee is told the time limits 
in advance. ^ . ' 

I? 



The specific tasks included on the performance tests are these: 
Chairside Assistin g 

Aajsisting with restorations: 
completing tray setups 

preparing the unit and chair 

<•. 

assisting with the preliminary examination of the patient 
assisting with the administration of a local anesthetic 
assisting with the placement and removal of a rubber c^am 
assisting with cavity preparation 
assisting with amalgam and composite restorations 
Surgical scrub and glove 
Patient Educa tion 

— ^ . — ^ . . . 

Instructing the patient in oral hygiene practices 
Dj^iKnostic Aids 

Taking vital signs: pulse rate, respiration rate, blood. pressure 

Dental radiography 

Charting from dictation 

Taking an alginate impression 
HenLa 1 Therapeut ics 

Placing and removing a periodontal dressing 

Coronal polishing 

Polishing an amalgam restoration 
Laboratory Procedures 

Pouring and trimming a study cast 
Some of the important characteristics of these tests are described below and" 
sunmarized in Table 2. 
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Tnblv 2. QuiractorlKt ICS of the Performance Tests* 

Procoa/product Type cf pntient 



Chalrslde Assisting: 
Assisting; with 
restoriit Iohb 



Surgical scrub and glove 

Patient FducUlf^n: 

Instructing n pntlcnt 
In oral hyj;lono 



Diagnostic Aids: 

Taking vital signs 

Dental radiography 



Chortlnp from- dictation 



Taking an alginate 
tnpresHlon . 

llierape ut ics : 

Placing and removing a 
periodontal dressing 

Coronal po I Ish i ng 



Polishing an amalgam 
restorat Ion 

Laboratory Procedures: 

Pouring and tr inning 
a study cast 



process 

process 
process 



both 
both 



(see "special 
features'') 



both 



both- 



both 



both 



manikin 



none 



actor 



product 



any person 
actual patient 



fictitious 

any person 

any person 
actual patient 

{ti;in Ikin 



person or 
pianlkln ^ 



Fxamlnee time 
(ninutes) 



60 (approK*) 

10 (approx.) 
20 

1 3 (approx • ) 

30 for exposure-, 
plus mounting 

15 (approx.) 

25 (approx • ) 



15 to place 
5 to remove 



10 



30 (»ipprox.) 



Special features 



Requires operator* 
Includes tray set-ups, 
local anesthetic* 
rubber dan* 



Patients' oral condition 
pr«$ented via chart and 
photos* 



Dcaminee does not process 
films* May use either 
bisoctlng or paralleling 
technique • 

Evaluation Is indirect; 
examinee uses own chart 
to answer questions* 

Two attempts pemlttcd* 
Includes wax bite. 



Examinee polishes two 
quadrants* ^atlent raust 
have any calculus removed 
prior to tost* 

Must maintain occlusal 
anatomy* 



Single pour In dental 
stone 0 
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Assisting with re storations * This test Is the longest and most complex 
of the performance tests. It Includes several tasks and Is the only one of 
the performance tests that requires a skilled person to take the role of the 
dentist. This person's behavior Is standardized for the te»t, by requiring 
him/her to work right-handed from a sitting position, receiving Instruments 
within a specified zone of transfer. The procedure for the test, Including 
the sequence of Instrument transfers. Is standardized and Is directed by one 
of the observers. The "patient" Is a manikin. Because the product of a 
restoration procedure Is tho responsibility of the dentist, rather than the 
auxiliary, the test In asslst-lng with restorations Is an evaluation of process 
only. It requires approximately an hour of, testing tine (not including set-up 
and clean-up). • ' 

Surgical scrub and glove. This test is a brief, straightforward test, 
requiring no patient and no special instructions. The test measures process, 
rather ^han producf. It requires less than ten minutes. 

Instructing a patient in oral hygiene . This test makes use of a testing 
technique called' che "programmed patient." The patient is a '^'patient-actor , " 
not necessarily a professional actor, but a person who has been specially 
trained for the role of patient in this examination. The patient-actor is 

f 

"pro^'.ranned" to respond in certain specified ways to the examinee's questions 
and instructions. This "program" provides a way to standardize the patient's 
behavior. Part of the patient's "program" involves the actual physical 
condition o.f the patient's mouth, which is presented to the examinee i-n the 
forn of dental charts and intraoral photographs. (The examinee is instructed 
not to inspect the patient-actor's mouth.) Pencils, paper, toothbrushes, 
dental floss; 'a hand mirror, and a dental manikin are provided. The examinee 

ERJC 



I 



ERIC 



-12- 



is responsible for deciding what to teach the patient and how to teach it. 
The only specification of goals for the patient education session is to 
"improve the patient's oral hygiene practices." Since there is no observable 
product, the test is an evaluation of process only. The examinee is allowed 
twenty minutes to examine the intraoral photos and other materials providing 
information ^bout the patient (dental chart, dental history, medical history) 
and to conduct the patient education session. 

Taking vital signs. This test is done on a person who need not be an 
actual dental patient. The examinee must take this person's pulse, . record 
the person's respiration rate, and take the person's blood pressure, using 
a stethoscope and blood pressure -uff. The accuracy of the examinee's 
measurements is determined by^ comparing then with measurements taken by the 
observers immediately before and after the examinee takes then. Each of the 
examinee's measurements is required to be within a specified tolerance of 
the nearer -of the two measurements taken by the observers or' of the average • 
of the two Observers' measurements. Th^ t^st emphasizes product — the 
accuracy of the measurements — but evaluates process also. It requires 
approximately fifteen minutes. 



Dental radiography. This test requlreS*the examinee to take dn actual 
Jf-ray film series on an actual dental patient. The series consists of 
fourteen periapical films and bite wings sufficient ^o cover the area. The 
examinee may use either bisecting or paralleling technique and may use any 

I 

type of film holder. The examinee does not process the films. Instead, the 
observer or another person familiar with the equipment at the test site 
processes the films. The exposures are made with double film packs. One set 
of films is returned to the examinee for mounting. The other set is sent to 
the patient's dentist. This test emphasizes product but includes an evalua- 
tion of process also. The completed x-ray films are evaluated as a group, 
rather than individually; the examinee receives ' credit for each specified 
anatomical feature demonstrated adequately at least once ^n the x-ray series.* 
The observers are provided with sample films showing examples of acceptable 
and unacceptable demonstration of each type of feature. These films have been 
solectod to represent borderline cases, so as to be of maximum benefit as an 
aid in scoring. Because the examinee may be taking the test on an unfamiliar 
x-ray unit> the films are not evaluated for density. The examinee- must 
complete the exposures in 30 minutes. There is no time limit for mounting 
the films. 

Charting from dictation . This test is different from the oth^er per- 
fornance tests in that the examinee's performance is not direccly observed* / 
Thoreforo, this tost can be adninistered to a group of examinees at the same 
time. The only limit on the size of the group is that all the examinees must 

tf 

*Thls portion of the test is adapted with permission from Arthur H. 
Wuehrnann, DMD: "Evaluation criteria for' intraoral radiographic film 
quality," .Jour nal of t>te American Dental Association . August 1974, 
Vol. 8y, pp. 345-352. 
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be able to hear the dictation. The test does not require the examinee to use 
a particular system of charting, and the examinee's chart is not evaluated. 
^ . Instead, the examinee must answer a series of questions on the basis *.of 
. his/her completed chart. The dictation is presented in the form of a tape 
recording. Between the two parts of the test; (charting from the tape 
recording and interpreting the chart) the examinees take other tests. The 
dictation takes approximately .ten minutes; the questions can be answered in 
five minutes. 

Taking an alginate impression. This test is done on a person who need 
not be an actual dental patient- The examinee must take alginate impressions 
.of both arches and a wax bite. The examinee may use either regular or fast- 
set alginate material (both are provided). The evaluation includes both 
process and product. There-is no tine. limit; the test requires approximately 
25 minutes. 

I 

Placing and removing a periodontal dressing . This test Is ^one. pn a 
person who does wot have an actual gingival wourid in the area where the 
dressing is to be placed. The observer uses a color transfer applicator to 
make a "wound" on the "patient's" gingiva. The test includes an evaluation • 
• of 'both process and product. The examinee is allowed fifteen minutes to 
mix, place, and trim the dressing and five minutes to remove it. 

Coronal polishing. This test is done on an actual dental patient. 
The patient must have all calculus removed before the test. The examinee 
is not expected to remove calculus, but is expected to remove extrinsic, 
stain and plaque. The test calls for the examinee to polish two diagonally 
opposed quadrants. The examinee may not use disclosing solution as an aid, 
although the observers use it in checking for the presence of plaque after 
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polishing after checking for evidence of laceration or abrasion of the 
patient's gingival tissue.* The test includes an evaluation of both process 
and product. The time limit for polishing both quadrants is twenty minutes. 

Polishing an amalga m restoration . This test is done on a dental 
manikin having a large unpolished restoration. The examinee is to polish 
the restoration so as to maintain the occlusal anatomy carved into the 

1 

restoration. The examinee is not expected to correct any irregularities in 
the unpolished restoration. The observers are provided with photographs of 
adequately and inadequately poli°shed restorations to serve as a guide in 
evaluating the polished restoration. The test includes an evaluation of ' 
both process and product. The time limit is ten minutes. 

Pour ing and trimtning a study cast . This test does not require a patient, 
but does require a completed alginate impression. (This can be the impression 
taken by the examinee as part of a performance test, or any other impression 
that Is available, including an impression taken on a^ dental manikin.) The 
examinee must pour the cast in dental stone, using a single-pour technique. 
Before the cast hardens, the "dxamlnee is to perform a "bench trim," using a 
lab knife to cut away excess material. After the cast hardens, the examinee 
must complete the trimming of the cast, using a model trimmer. This test is 
an evaluation of product only; the observer examines the completed cast, but 
dofs not observe the procedure. There is no time limit; the test requires 
approximately 30 minutes, not including the time spent waiting for the cast 
to harden (during which the examinee can take other performance tests). 
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* Disclosing solution makes this evidence difficult to see. In addition, 
the use of a disclosing agent before polishing would enable the examinee 
to get a perfect score on the test without polishing all s .rfaces of all 
teeth in the assigned quadrants, by oolishing only those surfaces showing 
plaque. 
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Table 3. Chronology of the Dental Auxiliary Examination Project 



August 1977 
October 1977 

January 1978 



February 1978 



March- July 1978 



August 1978 

November 1978. 
Decenber 1978 
January 1979 
February-March 1979 
March 1979 
April 1979 
May-July 1979 
August 1979 

August-September 1979 
September 1979 
October-November 1979 



Governnent Issues Request for Proposals 

Government Contracts with ETS to Develop 
Examination . " . 

First Meeting of Advisory Committee 



First Meeting of Working Committee; Selection 
of Subcommittees; Written Test Content and 
Performance Test Tasks Specified 

Writing of Questions; Review. and Editing by ETS 
,^taff 

Construction of Performance Checklists by Work 
Groups 

Meeting of Written Test Subcommittee * 



Meeting of Performance Test Subcommittee 
Second Meeting of Working Committee 
Second Meeting of Advisory Committee 
First Observer Training Workshop . 
Pretesting of Examination 
Thitd Meeting of Working Committee 
Second Observer Training Workshop 
Pilot Testing of Examination 
Fourth Meeting of Working Committee 
Writing of Final Report . 
Third Meeting of Advisory Committee 
Revision of Final Report 
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The Proficiency Examination was developed in a series of steps'^ over 
a period of nearly two years. Table 3 presents a chronology of the project. 

The Advisory Committee 

The original request for proposals for the examination project required 
the contractor (ETS) to "Establish an Advisory Committee ... to provide 
advice and to function as a sounding board to project staff." The Advisory 
Comm. I tee was to consist of representatives of several specified organiza- 
tions in the dental health professions, plus one member representing consumers 
of dental services. The professional organizations nominating members to 
the Advisory Committee were the following: 

American Dental Association 

American Association of Dental Examiners 

National Board of Dental Examiners 

American Dental Assistants' Association 

Certifying Board of the American Dental 
Assistants' Association 

American Dental Hygienlsts' Association. 

The Advisory Committee met three times. The committee's first meeting was 

held at the beginning of the project. At that meeting the committee made 

su^',gestions concerning the content of the erarainat Ion , suggested persons In 

the dental health professions to do the actual test development work, and 

suggested institutions that might serve -as test sites for tryouts of the 

examination. Between the Advisory Committee's 'first and second meetings, 

the American Dental Assistants' Association and the Certifying Board of 

the American Dental Assistants' Association withdrew their representatives 
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frjom the Advisory Committee.* Also, the representative of the American Dental 
Hygienists"" Association left her position with the Association and was replaced 
as the Association's representative, but continued as a member of the Advi&ory\ 
Committee* The Advisory Committee's second meeting was held shortly before 
the first tryouc cf the examination* At that meeting the committee reviewed 

c • 

the examination, suggesting some revisions, and recommended additional test 
sites for the second tryout of the examination* The Advisory , Committee's 
th-'-^d meeting was held at the conclusion of the project. At that meeting the 
committee reviewed an early draft of this report and made recommendations for ^ 



use of the examination* a 
The Working Committee 



4 ^ 

»t of«the Profici< 



The actual development of-^^he Proficiency Examination was done by a 
Working Committee of ten persons In the dental health professions, working 
with ET^ staff. Table 4 lists tTvT' members, af the Working Committee, their ^ 
occupational spjecialties , their places of residence, and the dates and 
place* of the meetings* For the initial phases of the project the Working 
Committee was divided into a Written Test Subcommittee and a t^erf ormance 
Test Subcopnittee, each consisting of one dental practitioff&r , one dental 
assisting practitioner, one dental assisting instructor, one dental hygiene 
practitioner, and one dental hygiene instructor. However, the full Working 
Committee reviewed both the written tests and t*e performance tests. 
Therefore, the entire examination must be considered to be the pro(luct of 
the full Working Committee. The members of the Working Commit tee -were 
selected from lists of names provided by the^^members of the Advisory Comr 
mittee 'and by the prof ess.fonal associations. In selecting persons for the 
Working Committee, ETS attempted to make sure that all geographic regions of 
the country were represented. 
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Table 4. The Working Coromittee 



Members ; * 

Dental Assisting Practitioner 

'r 

Dentalf Assisting Educator 
• J p. 

Dental Hygiene Practitioner 



De^ntal Hygiene Educator 
Dental Practitioner^ 

V 



* 4 



Written Test 
Subcoromittee 

Kathy Jesperson, CDA 
R&ck Va3ley, Iowa 

Suzanne C* Jones, CDA 
Chapel Hill, North Carolina 

Carolyn B. Hawkins, RDH 
Newport News, Virginia 

Sara Chinn-Karabasz , RDH 
Boston, Massachusetts 

Karl'W. Lange, DMD 
Lexington, Kentucky 



Performance Test 
Subcommittee 

Phyllis J. Shelton, CDA 
Ann Arbor » Michigan - 

Clara 0* Miera, CDA 
Albuquerque, New Mexico 

Cathy P. Turbyne, RDH 
Skowhcgan, Maine 

Evelyn R. Hobbs, RDH 
Seattle, Washington 

Rhane Wood, DDS 
Bartlesvllle , Oklahoma 



Meetings: February 10'-12, 4978; Princeton, New Jersey 

August 18-20, 1978; Alexandria, Virginia (Written test subcommittee) 

August 26-27, 1978; Albuquerque, New Mexico (Performance test subcommittee) 

November 18-19, 1978; Denver, Colorado 

March 24-25, 1979; New Orleans, Louisiana 

August 17-19, 1979; Boston, Massachusetts ^ 
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The full Working Committee met four times, a,nd each subcommittee met 
separately once. 'At the committee's first meeting the members divided into 
subconnnittees and drew up specif ications , for the written tests and the 

* * 

performance tests. The next meetings were subcommittee meetings to review 

f 

questions for the written tepts and early drafts of the printed materials for 
the performance tests. The second meeting of the full committee was devoted 
to a review of the written tests and the performance tests in preparation for 
the first tryout (the "pretest") of the examination. The third meeting was 
devoted to a review and revision of the tests on the basis of information 
gained from the pretest. The fourth meeting included a final review and 
revision of the examination. It also included a discussion of the issues of 
pass/fail cutoff scores and eligibility requirements, concluding with the 
committee's suggestions concerning these and other issues in the use of the 
examination. - . 

The test specifications for the Proficiency Examination were determined 
by the Working Committee, working from suggestions made by the Advisory 
Committee. The .specif ications were based on the task analysis documyit, 

"Competencies and Structural Elements Within the Classification of lental 

f 

Auxiliary Functions", which was provided by the Government.* Bott/the Advisory 
Committee and the Working Committee endorsed the premise that, although dental 

= ' # 

* This document Is a product of the Career Options Curriculum in Dentistry 
Project (Contract No. i-DH-34085. American Dental Hygienists' Association), 
whereby competency areas and structural elements of dental auxiliary 
functions were delineated by panels of experts who represented dental 
assisting, dental hygiene and dentistry in four geographic regions. An 
additional panel of experts, represented by dental assistant and dental 
hygienist education and practice, was subsequently asked to rate and 
rank the relative importance of each competency area and structural 
element according to specific criteria. This exercise resulted in 
producing the task analysis document that was used as the basis for the 
development of the Proficiency Examination. 
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assistants and dental hygienists do not all perform all of the tasks to be 
included in the examination, the level of proficiency required of a person who 
performs a task should be the same, regardless of whether the person is a 
dental assistant or a dental hygienist. 

t 

Development of the performance tests ^ 

Task specifications* The Advisory Committee met on January 7 and 8, ' 
1978 in Princeton, New Jersey to begin the project. They reviewed, the task 
analysis document and used it in making their suggestions tor the content of 
the examination. The committee's criterion for recommending tasks for the 
performance tests was that any task to be included in the performance tests 
should involve an important skill that cannot be tested adequately in a 
written multiple-choice test. Most of these skills involved the physical^ 
manipulation of dental instruments and materials, although some were communi- 
cations skills. 

Tlie Working Committee met on February 10, 11, and 12, 1978 in Princeton. 
New Jersey. At that meeting, the committee members divided into two sub- 
committees. Working from the Advisory Committee's suggestions and the task 
analysis document, the Performance Test Subcommittee developed specifications 
in the form of a list of tasks, including suggestions for the type of "patient" 
on which the examinee would perform each task: an actual patient, a manikin, 
or a "patient-actor" (i.e., a real person who is not an actual dental patient 
but has been trained to take that role in the examination). In general, the 
subcommittee's criterion for recommending one or another type of patient was 
to use a manikin for any tasks that could be tested adequately on a manikin 
and to use actual patients only if the examinee's skills could not be tested 
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adequately any other way. These specifications were not intended to be^ 
absolutely binding, but to serve as a guide for the subcommittee members who 
were to develop the performance tests in each competency area. 

One task that is conspicuous by its absence from the performance 
tests is the removal of calculus deposits. The Performance Test Subcommittee 
spent considerable time dispussing the problems involved in testing this 
skill and the amount of examinee time that such a test would require. They 
•finally decided not to include this task in the performance portion of the 
Proficiency Examination, but to recommend that the test developed as part 
of the American Dental Hygienists' Association's Clinical Evaluation Study 
be used to test this skill. 

* 

Review o^ existin g materials . As a preliminary step in constructing the 

> 

performance tests, ETS staff members reviewed materials produced by other pro- 
jects in the testing or training. of dental auxiliaries. These materials 
included the examination forms and booklets fromthe American Dental Hygienists' 
Association's Clinical Evaluation S tudy (DHEW Contract No. l-DH-44099), the 
Chairslde Assisting Skill E valuation (DHEW Contract No. 231-76-0031), and 
instructional materials from Project ACORDE (DHEW Contract No. 231-75-0407). 

TWO of the tasks in the ADHA Clinical Evaluation Study are included 
in the perTormance tests of the Proficiency Examination: dental' radiography 
and coronal polishing. However, the members of the Performance Test Sub- 
committee preferred an approach to the measurement of these skills that 
included an evaluation of process as well as product. On the other hand, they 
decided to rely on the ADHA Clinical Evaluation Study for a performance test 
in calculus removal, rather than to develop a performance test for that task. 
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'^^ Chalralde Aaslstlng Skill E valuation (CASE) materials Include 
two large collections of performance checklists, one "designed for use by 
students and faculty ia the institutional (campus facility) -setting," and 
the other "designed for use during the student's extramural clinical 
experience assignment." The CASE materials include checklists for a great .. 

4 

many tasks. Some of these tasks are included in the Proficiency Examination, 
in the: performance test in assiat-lng wtth"" restorations. The influence of 
the CASE materials can be seen in portions of the performance checklist for 
this test. 

The Project ACORDE mater^ala are instructional materials, not evalua- 
tion materials. However, they contain detailed step-by-step descriptions 
of the proper procedure for performing several important tasks done by 
dental auxiliaries. The Performance Test Subcommittee members found them 
quite helpful in developing the performance checklists. 

Developm ent of performance checklists . 'The initial development of the 
performance tests was accomplished by small work groups. Each work group 
consisted of two Performance Test Subcommittee members and two ETS staff 
members (and, in one case, another Working Committee member who was not on 
the Performance Test Subcommittee) . Table 5 shows the committee members 
ser\ Ing on each work group and the dates and locations of the work aessions* 
The work sessions were structured in an informal interview format, with 
an ETS staff member serving as interviewer. The ETS. staff member led the 
committee members in talking through the task to be tested, step by step, 
using questions such as the following: 

"What does the auxiliary do next?" 

"What kind of mistake might the auxiliary 
make in doing this step?" 

32 , ■ 
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Table 5. Work Groups for Developmeat of the PerformancQ Tests. 



Test Modules 

Chairside assisting. 
Laboratory procedures 



Diagnostic aids 
Patient education 



Dental therapeutics 



Work Group Members 

C. Miera 
P. Shelton 



E. Hobbs 
C. Turbyne 



S. Chinn-Karabasz 
C. Miera 
C. Turbyne 



Work Sessions 

March 7-8, 1978 

Ann Arbor, Michigan 

April 10-11, 1978 
Ann Arbor, Michigan 

March 17-18, 1978 
Chicago, Illinois 

May 31-June 1, 1978 
Seattle, Washington 

May 21-23, -1978 

Boston, Massachusetts 
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"How can a person observe whether the 
auxiliary has done this step right or 
wrong?" 

In this way, the work group would identify "checkpoints": specific observable 
actions to be done in a specified way by the examinee. Ideally, each check- 
point was to correspond to one possible error (made or avoided) by the 
examinee. After identifying the -ckeckpoints to be Observed, the work group 
members would decide what equipment and materials to specify, and \A\at 
instructions to give the examinee and the observer. 

After each work session, the ETS staff members used their notes' from the 
session to draft a performance checklist for each task or group of tasks. The 
performance. checklist consisted of concise statements of the checkpoints for 
the task, with spaces for the observer to check "yes" or "no" for each check- 
point. The checklists also contained Instructions to the observer for 
directing the procedure. After drafting the performance checklists and 

Other tost materials (equipment lists, additional instruction3 for observers, 

« 

examinees, instructions, etc.), the ETS staff sent the test materials to the 
members of the work group, who indicated the necessary corrections by phone or 
mall. The ETS staff members then sent corrected first drafts to all five 
members of the Performance Test Subcommittee for review and comment. 

In constructing the performance test In dental radiography, one 
of the work group members suggested basing the evaluation of the finished 
radiographs on a previously published evaluation form** ETS received 
permission from the author to' adapt this evaluation form for use in the 
Proficiency Examination. 



Arthur H. Wuehrmann, DMD, op> cit» 
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In the course of developing the perf ormaflQe tests, the work groyps 
suggested several changes in the task specifications* In each case, the 
other members of the Performance Test Subcommittee concurred in the changes* 
liany of the changes involved the deletion of tasks that presented practical 
difficulties or required amounts of testing time that, in the subcommittee 
members' opinion, outweighed the desirability of including these tasks in 
the hands--on performance portion of the examination • 

Review by consultants * At the first meetings of the Advisory Committee 
and the Working Committee, committee members suggested that ETS contact thf 
dental instructor who had served as evaluation specialist for the American 
Dental Hygienists' Association's Clinical Evaluation Study (DHEW Contract No* 
l-DH-44099) .* This instructor in turn recommended that ETS consult one 
of his colleagues, also a dental instructor and an expert in dental auxiliary 

Utilization.'** Both these dental instructors served as consultants, to 

« 

review the* perf ormance tests from the dental instructor's point of view* 
Their role in the project was purqly advisory; all decisions involving the 
content and format of the performance tests were made by the members of the* 
Working Committee* 

Review by subcommittee * The Performance Test Subcommittee met in 
Albuquerque, New Mexico on August 26 and 27, 1978 to review the performance 
tests* Approximately a week before the meeting, each subcommittee member had 
received in the mail a copy of' the performance checklists and other printed 
materials (observer's instructions, patient-actor's instructions, etc*)* With 
these materials was a letter asking the subcommittee member to review the 



i 

* John Eisner, DDS, New Jersey Dental School* ' 
** Richard Montgomery, DDS, New Jersey' Dental School* 
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materials with the following points*^ mind: 

1. Does anything need to be added to the checklists or instructions? 

2. Is there anything in the checklists or instructions that should 
be taken out? 

3. Is there anything in the checklists or instructions that needs 

to be rewritten (e.g., anything that is not completely correct or 
not clear to the reader)? .v 

^* Are there any checkpoints or groups of checkpoints that are 

not essential and could be deleted (to s *ve time or to lighten 
the load on the observer)? 

5. Are there any checkpoints that are so important that no examinee 
should Tie permitted to miss one and still pass the test? 

6. If the test takes too long to administer, what can be taken out? 
The subcommittee reviewed the jnaterials and made several minor changes. 
Rev iew by the full Working Committee . The full Working Committee met in 

Denver, Colorado on November 18 and 19, 1978 t6 review both the written tests 
and the performance tests. Before the meeting, each membeir of the committee 
had received in the mail a copy :f the performance test materials, with a 
letter similar to the one sent to the Performance Test Subcommittee. The 
conunittee reviewed the performance tests and made several specific changes, 
including the deletion of one task (preparation of an Impression material). 

Review by the Advisory Committee . The Advisory Committee met in 
Alexandria, Virginia on December 1 and 2, 1978. The primary purpose of che 
Advlfjory Coniniittee meeting was to enable the committee to review and comment 
on the examination. Two men.bers of the Working Committ3e, both dental 
assistants, attended the meeting, to help answer the Advisory Committee's 
questions and to make sure the Advisory Committee's suggestions were trans- 
mitted accurately to the Working Committee. The Advisory Committee offered 
several specific suggestions for the performance tests. ETS relayed these 
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suggestions to the members of t\te Working Committee most directly .involved, 
and, with thefr approval, most of these suggestions" w6re incorporated into 
the examination:. 

Selection of pretest and pilot test sites * The project included two 
trial administrations of the examinations. The first, called the pretest , 
was to identify any unforeiseen . difficulties in administering the examination 
and to provide, data for use in revising the examination. The ..second, called 
the pilot test , was for. the purpose of collecting data on a nationwide sample 
of examinees. The test sites were all colleges with programs in dental 
assisting or dental hygiene — in most cases, both. They were suggested by 
members of the Advisory Committee and the Working Committee and were selected 
on the basis of their facilities, personnel, and willingness to participate. 
The pilot test sites were also selected for geographic distribution. 

The personnel necessary to administer the examination at each pretest o 
jSilot test site included a coordinator, at least two observers, an operator, 
a patient-actor, and a dentist-on-site. The coordinator was responsible for 
organizing all testing activities , obtaining the necessary equipment* and 
materials, and arranging for examinees and patients. The observers were 
responsible for observing and recording each examinee's performance in the 
performance tests, for preparing the operatory for each examinee, and, in 
those performance tests that involved a live patient, for assur^ing the safety 
of Che patient. The observers for the pretest and pilot test were all 
instructors in dental assisting or dental hygiene. The operator was needed 
only for the performance test in chairside assisting, in which the operator 
places restorations in a manikin, using the rubber dam, while the examinee 
assists. The patient-actor was needed only for the performance test in 

37 
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patient education. The ,dentist-Qn-site was responsible for performing a 
preliminary examination on each actual dental patient and was to be present 
during the performance tests involving an actual patient*, to deal with any 
emergency that night arise. Some of these different functions could be 
perforned by the same person. For exkmple, at many test sites the' coordinator 
served as one of the observers and the dpntlst-on-site served as the operator. 

Five institutions served as test sites for bofch the pretest and pilot 
test. They were located in New Jersey, North Carolina, Michigan, Illinois, 
and Iowa. An additional seven Institutions served f s. test sites for ^he pilot 
test. Two of these were neighboring institutions, one with a dental assist- 
ing program and one with a dental hygiene program; they combined resources to 
sosvvv as a single test site. These additional test sites were located in 
Illinois, Texas, New Mertico, Arizona, California, and Washington. Thus, the 
eleven pilot test sites included twelve institutions: one in the Northeast, 
one in the Southeast, four in the Midwest, three in the Southwest, and three ' 
in the Far West. Table 6 lists these institutions. 

Training, of obse rvers for the pretest . The observers and the operators 
for the pretest attended a two-day training workshop on January 20 and 21 at 
Dos Moines Area Community College, Ankeny, Iowa. The workshop was conducted 
by the coordinator for that test site and an ETS staff member. The workshop 
consisted mainly of practice sessions, at which the observers and operators 
practiced administering the tests to dental assisting students who served 
as examinees. Interspersed with the practice sessions were debriefing 
sessions, at which the observers and operators discussed the problems they 
had encountered and made suggestions for overcoming those problems. One 
member of the Working Committee attended, to help th^ ETS staff respond to 
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Table 6. Pretest and. Pilot Te^t Sites. 



Pretest and r'lot Test: 

Des Moines Area Community College* 
Ankeny , lou^ 

Guilford Technical Institute 

Jamestown, Nortii Carolina 

Middlesex County College 
Edison, New Jersey 

Southern Illinois University ^ 
Carbondale^, Illinois 

University of Detroit 
Detroit, Michigan 



Pilot Test Only: 



^ 0 



Cabrillo College* 

Apt OS, California^ 

Edmonds Community College**^ 
Lynwood, Washington 

Loyola University 

Maywood, Illinois 

Maricopa Technical Community College 
Phoenix, Arizona 

'Shoreline Community College ** 
North Seattle, Washington 

University of New Mexico 

Albuquerque, New Mexico 
* 

University of Texas . 

San Antonto, Texas * . 



*Site of observer training workshop 

**These two institutions combined to serve, as a 
single test site 
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questions from the observers and operators and to advise ETS as to which of 
their suggestions could be adopted directly (e.g., adding a diagram t/ aid 
the observer) and which needed to be discussed with the Working Committee. 

During pretesting (and pilot testing) of the performance tests each / 
examinee was to be observed by two observers, workirig independently. The 
olpservers were not to look at each other's resprnses until after they had 

line was added at the end of each performance checklist, calling for the 
observer's overall judgmetit of tKe exarainae's performance as adequate or 
inadequate. The observers were cautioned that "adequate" was not to be 
Interpreted as anything more than merely adequate; the observer's judgment 
was -to indicate whether the examinee's performance would be acceptable for 
an entry-level dontal auxflia^y in a dental office or clinic. In addition, 
the observers were urged to write comments" q.d the checklists to indicate any 
examinee errors or administrative problems that had not been anticipated in 
the performance checklists and instructions. 

PrctestijTg. During February and March of 1979 the examination was 
administered at the five pretest sites. Most of the pretest examinees were 
students in dental assisting or dental hygiene.. They were paid for their 
participation. The written tests were administered first, in a single half- 
day session. The scheduling plan for the performance tests varied frcm one 

t 

test site to another. Some test sites tested only one examinee at a time; 
others trained additional observers and tested two examinees at a time. In. 
most cases, the same examinees took all the tests, but in some cases different 
examinees took different performance tests. (In some cases the test site 
coordinators could not find examinees who had a basic knowledge of chairside 
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asslatlng and could also perform all tasks in the performance tedts of thie 
therapeutics module.) Table 7 shows the number of examinees taking each 
performance test during pretesting. 

One^ day of performance testing at each pretest fiite .'yas observed by a 
member of the Worklng^Cominl,tte8« In this way five members of the committee 
gained first-hand 'knowledge of the tests in actual use. T^'ble 8 lists the 
Working CommHtee me^mbers who attended tt.e observer training and^the pre- 
testing. ' . 

The pretest data for the performance tests were Analyzed separately for 
.each of the tests, the data were analyzed in two different ways; one con- 
centrated on reliability and the ^her on validity. ^'^e reliability analysis 
involved a comparison of the two independent observations of each examinee on 
each checkpoint. Any checkpoint for which the agreement between observers was 
less than 85 percent (of the examinees) was i4entified for special review, to* 
detect ambiguities of phrasing, .difficulty of observation, and other possible 
problens. The validity analysis was based on a division op the examinees 
into "adequate" and "inadequate" groups on the basis of thp observers' overall 

i 

judgment. Any checkpoints missed by a high proportion of jthe examinees in 
the "adequate" groups were identifie:. for special review cj^ncentrat ing on the 
importance of the checkpoint and on tie possible acceptability of accomplish- 
ing the task in ways not specified in the checkpoint. ^ 

In addition to the formal data analysis, LIS staff tabulated the com*- 
ments written on the checklists and recorded the additional comments furnished 
by coordinators, observers, and Working Committee members who observed the 
testing. (Two observers from the New Jersey test site traveled to ETS and 
spent an entire afternoon discussing their comments and suggestions with ETS 
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Table 7. Number of Examinees Taking Each Performance T€at During 
. Pretesting of Examination. 



•V^ Assisting with restoratio;is » 21 

Surgical scrub 21 

, Oral hygiene instruction 18 

•Taking of vital signs 24 

Dental radiography 2A 

- Charting from dictation 22 

Alginate impression 22 

Periodontal dressing 20 

Coronal polishing 21 

Polishing amalgam restoration 21 

Study cast 21 
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^able 8. WorVing Committee Members Attending Observer Training and 
Pretesting of Examination* 

Pretest observer training 

Des Moines Area Community College -'T^'^Jespersen^^ 
Pretesting 

Des Moines Area Community College - K. Jespersen 
Guilford Technical Institute S* Jones 
Middlesex County ^lollege - C. Turbyne 
Southern Illinois University - C. Miera 
University of Detroit - ?• Shelton 

Pilot teat observer training 

Cabrillo College - £• Hobbs, C. Miera 
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staff.) These comments Indicated several minor problems and one major problem 
with the performance tests. The major problem involved the test in assisting 
with restorations. Early versions of this test called for the operator to 
direct the procedure, working from memory with the aid of a printed script. 
All five pretest operators found this procedure difficult and confusing. They 
suggested having one of the observers direct the procedure from instructions 
printed on the checklist, a suggestion that was subsequently adopted by the 
Working Committee. Of the other suggestions, some Involved adding, deleting, 
or rephrasing individual checkpoints; others involved procedures for adminis- 
tering the tests. 

Revision of tesf: materials by the Working Committee . The Working 
Committee met on March 24 and 25, 1979, in New Orleans, Louisiana to review 
the pretest results and revise the examination. The committee made many 
^revisions in the performance tests. * Most of the revisions tended to simplify 
the test materials and procedures. Many checkpoints were deleted, and some 
checklist formats were changed to make the observer's task easier. The pro- 

c 

ceduro for the test in assisting with restorations was placed under the 
direction of the observer rather than the operator. 

Trai ning of observers for the pilot test . The observers and operators 
from the pilot test sites that had not also been pretest sites attended a 

twcwday training workshop on April 21 and 22, 1979 at Cabrl^lo College, 

\ 

Aptos, California. The workshop was conducted by the test sl'te coordinators 
for Cabrillo College and for Des Moines Area Community College (where the 
previous workshop had been held) and an ETS staff member. Like the previous 
workshop, this one consisted mainly of practice sessions interspisrsed with 
debriefing sessions. Students in dental assisting and dental hygiene served 
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as examinees. Two members of the Working Committee were present, Including 
one who was also a pilot test site 6oordlnator and observer. As expected, 
the revised procedures went much more smoothly, and the observers did not 
encounter any serious problems In administering" the tests or observing the 
examinees' perforince* Only a few minor changes in the performance test 
materials were made as a result of this workshop experience. 

Pilot testing. During May, June, and July of 1979 the examination was 
pilot tested at the selected pilot test sites* The pilot test was admin- 
istered to a total of 115 examinees: 58 dental assistants and 57 dental 
hygienists. They were paid for their participation. Only the' dental 
hygienists tc'«k the performance tests in the therapeutics module (periodontal 
dressing, coronal polishing, and polishing an amalgam restoration). Examinees 
at some of the test sites did not take the performance test in the taking of 
an alginate impression, because of state laws prohibiting auxiliaries from 
doing this procedure. In addition, some of the hygienists refused to attempt 
the performance test in assisting with restorations, on the grounds that they 
had not had any training or experience to prepare them to perform this 
function* 

Table 9 summarizes the professional experience of the pilot test 
examinees. Most of the examinees had only one or two years' experience in 
their fields, but several had more. One of the dental assistants was a dental 



assisting instructor, and eleven of the dental hygienists were (or had been) 



dental hygiene instructors^ eight of them for at least three years* None of 
the dental assistants had any experience in dental hygiene, although one had a 
year of instruction. However, fifteen of the dental hygienists had been dental 
assisting practitioners, and three had been dental assisting instructors. 
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Table 9. Summary of Professional Experience of Pilot Test Examinees^ 



Professional Credentials* 
CDA 

RDA 

RDH 

Instruction in Dental Assisting 
None 

1 yeor 

2 or more years 

Instruction in Dental Hygiene 
None 

1 year 

2 years 

3 years 
' 4 years 

Ey,perience as Dental 
Assisting Practitioner 

None 

1 year 

2 years 

3 years 

4 years 

5 years 

6 or more years 



Dental 
Assistants 



53 
6 
0 

0 
50 
8 

57 
1 
0 
0 
0 



0 
14 
20 
10 
5 
2 
7 



Dental 
Hyglenists 



5 
0 

'57 

50 
5 
2 

0 

0 

51 ■ 
2 
4 



42 
5 
3 
1 
1 
1 
4 



*Some persons held more than one professional credential. 



Bental 
Assistants 



Dental 
Hyglenlsts 



Experience as Dental 
Hygiene Practitioner 

None 

1 year 

2 years 

3 years 
A years 

5 years 

6 or more years 

•% 

Experience as Dental 
Assisting Instructor 



58 
0 
0 
0 
0 
0 
0 



5 
23 
16 
2 
1 
2 
8 



None 


57 


54 


1 year 


0 


2 


2 years 


0 


1 


3 years 


0 


0 


. 4 years 


0 


0 


5 years 


1 


0 


as Dental 
Instructor 




« 


None 


58 


46 


1 year 


0 


3 


2 years 


0 

• 


0 


3 years 


0 


2 


4 years 


0 


3 


5 years 


0 


2 


6 years 


0 


1 
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The pilot test data from the performance tests was analyzed in the same 
way as for the pretest, to detect any performance checkpoints that still caused 
problems of inter-observer agreement and to identify checkpoints missed by a 
large proportion of the examinees whose performance was judged adequate by the 
obseivers. Perfofnance test data from assistants and hygienists were analyzed 
together. Some additional analyses were done on the pilot test data from both 
the written tests and the performance tests, and these analyses will be 
reported in the following pages. 

The pilot testing revealed substantial differences between test sites • 
"in the amount of time required to set up and administer the performance tests 

r- , 

and in the cost of equipment and materials that had to be purchased for these 

< 

tests. The differences in the. time required to administer the performance 
tests were not as great as the df ferences in set-up time. The differences 
between test sites in set-up time were so great that any general statement 
as to the amount of set-up time required for each test would be misleading 
for a large proportion of the test sites. 

Uevolopment of sc oring formulas . The development of scoring formulas 
for the performance tests was a three-stage process. Before the pilot 
testing of the examination began.* each member of the Working Committee 
received in the nail a set of the performance checklists, with a letter 
asking for a series of judgments made by the following procedure; 

1. Identify any checkpoints that you think should be deleted. 
Give them a weight of zero. (There may or may not be any 
such checkpoints.) 

.2.. Identify any checkpoints that you consider so important 
that any examinee who misses even one such checkpoint 
should receive a failing score for that portion of the 
examination. Indicate these checkpoints with an asterisk, 
instead of giving them a numerical weight. (Again, there 
may or may not be any such checkpoints.) 

48 



3* Identify the least Important of the remaining checkpoints* 
Give each of them a weight of one. 

4# Identi the most important of the remaining checkpoints*^^ 
Decide how. many points each of them should be worth* ThatE^^ 
is^ how many of the low- importance checkpoints should equal 
one of the high-Importance checkpoints? (One way to makeX 
this decision is to imagine an examinee who missed only one 
checkpoint — the most important* How many .of the least \^ 
important checkpoints would a second examinee have to miss^ 
in order to deserve the sane score as the first examinee?)^ 

5* Assign weights to the remaining checkpoints by comparison ^ 
with the most and least important* (You can .!se the same 
sort of reasoning as £or step 4«) 

The Working Committee members wrote their suggested scoring weights/^ 
on the performance checklists and returned them to ETS* (Not all committee 
members suggested weights for dlX the performance tests; for example, none of 
the dental hygienists suggested scoring weights for the test in assisting with 
restorations*) 

When the. checklists arrived at ETS, the committee members' suggested' . 
scoring weights for each test were collated on- a single master copy of the 
checklist* An ETS staff member reviewed the weights and assigned a single 
consensus weight to each checkpoint* ETS then printed copies of the perform- 
ance checklists marked with these consensus weights (labeled "preliminary 
scoring weights") and sent them to the members of the Working Committee* With 
the performance 'checklists was a letter asking the committee members to review 
the'^preliminary scoring weights in preparation for the final meeting of the 
Working Committee* The committee members were to note any checkpoints they 
felt had been weighted coo heavily or not heavily enough* The committee's 
review and revision of the scoring formulas at t^e meeting was the third and 
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final stage of the process. Table 10 shows the number of checkpoints and the 
range of scoring weights in each of the performance tests (except charting 
from dictation, which did not Include, a performance checklist). • 

» 

Final review bV the Woflcing C ommittee. The Working Committee met on 
August 17, 18, and 19, 1979 in Boston^ Massachusetts. In r.eviewing the 
performance tests and scoring formulas, the committee gave special considera- 
tion to those checkpoints that showed poor intfcr-observer agreement (less than 
85 percent)^ and those that were missed by a large proportion of the examinees 
whose performance was judged adequate. The committee changed the wording of 
several of the checkpoints that showed poor inter-observer agreement. They 
also deleted some of the checkpoints and made several adjustments to the 
scoring formulas. - 

Scoring of the p erformance tests . Each performance test (except for the 
tost in chartina from dictation) was scored by a three-stage procedure. First, 
the scoring formulas were applied to each checklist separately. Then the 
resulting scores were converted to percentage terms, expressing the score as a 
perconta^c of the total possible score. This percentage scoring was "necessary 
booause of checkpoints left unmarked by the observers and because a small 
number of the performance checkpoints did not apply to all examinees (because 
of d*i<fferences in equipment, patients, etc.). Finally, the scores computed 
frcm the two observers' checklists for each examinee were averaged to produce 
the examinee's score. 

Table U presents the distributions of the pilot test examinees' scores 
on the performance tests. The scores on the test in assisting with restora- 
tions arc quite high because the performance checklist includes checkpoints 
for a large number of errors that are rarely made, particularly in selecting 



50 



-42- 



Table 10. Numl>er of Checkpoints and Range of Scoring Weights for Performance 
Tests. 



Number of 



Range of 



. \ 



Test 


Checkpoints 


scorinj^ 


weights 


Assisting with restorations 


382* 




10 


Surgical scrubs 


13 


1 - 


10 


Oral hygiene 'nstruction 


21 


1 - 


5 


Vital signs 


27 




10** 


l^adiography 


155*** 




10 


Alginate impression 


65 




10 


Periociontal dressing 


29 




20 


Coronal polishing 


3i 




20 


Polishing amalgam restoration 


29 ' 




10 


Study cast 


37 




10 



* Includes several checkpoints for each Instrument exchange. 

** Reported measurements must be correct, within specified tolerances; 
otherwise the examinee fails the test. 

*** Each feature to be demonstrated is counted here as a separate checkpoint 
Nuuber shown is maximum; actual number depends on patient. 
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Table 11. Distributions of Examinees' Scores on Performance fests (average of 



two observers). 



ERLC 



lesl Scor«: 



Assisting with 
Restorations 



Sor(|ical 
Scrub 



ffral 
Hygiene 
Instruction 



" Alginate Periodontal 
Signs ^ Radiography Impression Dressing 



PoXishing 
Coronal Amalgam Study 
Polishinq Restoration Caftt 



96 - 100 


38 (48fo) 


3 { 4''o)"' 


1 ( IS) 


33 (30S) 


Q 




6 \\2%) 


8 (15S) 


7 (125) 


3 ( 3») 




91-95 


23 (515.) 


' 3 ( 4%) 


3 ( 4S) 


7 ( 6S) 






4 ( 8*) 


2 ( 4S) 


7 (125) 


7 ( 65) 




86-90 


14 ( 18*^1 ) 


7 { 6fi) 


6 ( 5S) 


28 I 2^%) 






o ( \6%) 


7 (13S) 


6 (1*15) ; 


6 ( 55) 




81 - 8S 


1 ( 1?.) 


9 ( 8'^) 


10 ( 9S) 


14 ( 12;^) 






7 (14S) ^ 


9 (17S) • 


10 (165) 


A ( 45) 




76 - 8U 


1 ( 1%) 


9 ( 8S) 


12 (IIS) 


7 ( 






15 (29S) 


9 (17S) 


16 (285) 


9 ( 85) 




71 - 73 


1 ( 


3 { 4%) 


. 13 (13S) 


14 ( M%) 






2 ( 4S) 


6 (125) 


5 ( 95) 


17 (155) 


• 


66 - 70 


0 


3 ( 3S) 


13 ( 13S) 






1 ( |3») 


2 ( 4'«) 


3 ( 6S) 


3 (55) ■ 


13 (125) 




61 - 63 


0 


3 ( 45i) 


8 ( 7S) 


1 ( IS) 


13 (IIS) 


* 0 . 


2 ( 4S) 


2 ( 45) 


1 ( 25) 


12 (115) 




^ - • 60 


0 


3 ( 4S) 


12 (IIS) 




14 (12S) 


2 ( 3S) 


0 


S (105) 


0 


11 (105) 




m 


0 


13 (IIS) 


-8 ( 7S) 


Q 


4 ( 3S) 


0 


3 6S) 


1 ( 25) 


0 


7 ( 65) 


1 


46 - SO 


0 


9 ( 8S) 


7 .( 6S) 


0 




n 
U 


0 . 


0 


0 


8 ( 75). 




41 - 43 


0 


17 (14S) 


4 ( 4S) 


0^ 


3 ( 4S) 


0 


2 ( 4S) 


0 


1 ( 25) 


7 ( 65) 




56 - 40 


0 


10 ( 9S) 


3 ( 4S) 


Q 


2 { 2S) 


0 


2 ( 4S) 


0 


1 ( 25) 


2 C 25) 


* 1 


31 - 33 


0 


3 ( 4!S) 


3 ( 3S) 


Q 


1 ( IS) 


0 


0 


0 


0 


2 ( 25) 




0-50 


0 


■ ■ 8 ( 7S) 


2 ( 2S) 


Q 


0 


0 


0 


0 


0 i 


4 ( 45) 




Total Number 
























of Examinees 


80 




113 


115 




73 


31 


52 


57 


112 




Average Score 


92.8 


38.1 


64.8 


85.6 


67.0 


85.7 


78.3 


78*7 


82.0 


63.8 




Standard 


3.3 


21.4 


16.2 


10.9 


12.6 


7.8 

♦ 


14.2 


12.6 


11.4 


19.7 


• 



52 



53 
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and delivering hand ins truments • Each instrument exchange involves se^veral 
possible errors, and, since these are not all equally important, the perform- 
ance checklist contains a separate checkpoint for each* There is a great deal 
of variation in the examinees' scores on some of the performance tests,^/ 
particularly the surgical scrub and the study cast. 

The performance test in charting from dictation was different from the 
other performance tests in that the examinee's performance was not directly 
observed* Instead, the examinee was to answer a series of multiple-choice 
questions based on his/her own chart* Table 12 shows the distribution of 
scores on this test* 

Inter-observer reliability of the performance' tests . The use of two 
independent observers in the performance tests enabled ETS to investigate 
the inter-observer reliability of scores on those tests. This type of 
reliability can be described in several different ways, not all of which 
are relevant for criterion-referenced tests.* One type of information 
that is relevant for criterion-referenced tests is thq size of the dif- 
ferences between observers' scores for the same examinees. Table 1.3 
presents this information. Since the scores are expressed as percentages 
of' the maximum possible score, the differences in Table 13 are in lerms of 
percentage points. (For example, if the two observers' checklists for the 
feane e^^inqe yield scores of 80% and 8^%, the size of the difference is four 



* Measures such as the intraclasf; correlation coefficient, which depend 
or. the amount of variation between examinees in their true level of 
performance, are not relevant to criterion-referenced tests. 
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Table 12. Distribution of Scores on Performance Test in Charting from 
Dictation. 



Test score 
12 
U 
10 

9 

8 

7 

6 

0-5 



Numbe.r of Examinees 
55 (48%) 
38 (33Z) 
14 (12%) 

4(3%) 

2 ( 2%) 

0 

2 ( 2%) 
0 



Total 



115 



55 



's 



Table 13. Distributions of Inter-Observer Differences on Performance .Testes. 



Assisting with Surgical 
Restorations Scrub 



Difference 
size: 

0-5 

6 - 10 

11 - 15 

16 - 20 

21 - 25 

26-30 

31 - 35 

36 or more 

Total Number 
of Examinees 

Average 

difference 

size 



76 (95%) 
3 ( U%) 
0 

1 ( IS) 

0 

0 

0 



80 



Oral 
Hygiene 
Instruction 



Vital * . Alginate 

Signs Radiography Impression 



Periodontal Coronal 
Dressing Polishing 



59 (515) 5ft (ft7S) 89 (77S) 83 (72S) ft8 (66S) 29 ( 57S) ft3 (83S) 



16 (US) 25 (205) , 10 ( 95) 17 (155) 
7 ( 65) 22 (195) 11 (105) ft ( 35) 



l.ft 



75) 



A ( 35) 



7 ( 65) 



3 ( 35) 



11 (105) 



115 



10,7 



6 ( 55) 



3 ( 35) 



ft ( ftS) 



1 ( 15) 



113- 



7.3 



2 ( 25) 9 ( 85) 



1 ( 15) 1 ( 15) 



2 ( 



25) 



115 



1 ( 15) 



115 



19 (265) 

3 ( ft5) 

2 ( 35) 
0 

1 (15) 



3.6 



3.9 



73 



ft.O 



8 (165) 
ft ( 85) 
2 (ft5) 



1 ( 25) 



2 (ft5) 



51 



8.0 



A ( 85) 
2 ( 45) 
1 ( 25) 



5 (105) 1 ( 25) 



1 ( 25) 



52 



3.6 



Vjlishihg 
Amalgam Study 
Restoration Cast 



ftO (70S) 64 (575) 

7 (12Si) 16 (145) 

4 ( 7!6) 20 (185) 

1 ( 25) 6 ( 55) 

4 ( 75) 4 { 45) 



1 ( 25) 
0 



57 



4.5 



1 ( 15) 



1 ( 15) 



112 



6.1 



I 



56 



57 
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percentage points.) The inter-observer rellabiUty .of most of the tests 

r 

t . * ^ 

appears fairly good. However, the surgical scrub, oral hygiene instruc- 
tion, and periodontal dressing seem to present some problems in inter- 
observer reliability. Some of the revisions made after pilo.t testing (change^ 
in- phrasing and sequencing of checkpoints) may improve the inter-observer 
reliability of the performance tests. However, in some cases the important 
points may simply be difficult to observe — e.g., avoiding cross-contami- 
nation during the surgical scrub. In other cases, there may be no practical 
way to avoid the need for judgment by the observer — e.g., in deciding 
whether or not an examinee has explained a particular point to the. patient 
during oral hygiene instruction. .. " ' 

f 

The data in Table 13 are totals for all the test sites combined. 
However, there were substantial differences between test sites in the inter- 
obscrver reliability of the performance tests. Table 14 shows the average 
difference between observers for each test at each inatj^^drT involved in 
the pilot testing of the examination, "a small number indicates that the 
Intor-ghserver reliability of that test at that institution was good; a 
large number indicates poor inter-observer reliability. The numbers in 
Table 14 reveal some interesting facts. First^ at some institutions the 
ob«crvurs weru able to observe ail the performance tests reliably.' Second, 
sou,- Institutions seem to have had inter-observer reliability problems 
for some of the tests but not others. Third, no institution seems to have 

* 

had serious reliability problems with all the tests. The differences 
between institutions in inter-observer reliability may be associated with 
differences in the ways they administered the tests. If so, the inter- 
observer reliability of the performance tests could be potentially much 



5S 



f 



5.9 



Table _14. Ayerige lnter-Obgervet Difference for Performance Tests at Each Testing Institution. 

Testing Institution 



Performance Test: 


1 


2_ 


3 


4 


5 


6 


_7_ 


_8_ 


9 




_1JL 


__12_ 


Assisting with restorations 


1.6 


0.3 


1.1 


2.0 


2.8 


1.5 


1.2 


0.8 


1.5 




1.3 


0.8 


Surgical scrub 


18.7 


0.0 












9.1 


7.9 


2.4 


18>.2 


22.5 


Oral hygiene instruction 


8.7 


1.6 


8.4 


8.9 


10.1 


2.6 


4.5 


14.7 


7.0 


3.2 


10.1 


5.2 


Vital signs 


3.7 


0.1 ' 


2.6 


1.4 


5.5 


2.7 


1.8 


4.6 


8.0 


0.0 


4.3 


6.5 


Radiography 


4.0 


0.2 


6.4 


12.9 


5.3 


2.7 


1.3 


5.1 


3.2 


0.0 


3.3 , 


. 0.9 


Alginate impressing 


3.8 




2.7 


3.7 


A 


* 


4.2 


2.8 


A 


0.6 


6.1 


6. J 


Periodontal dressing 


10.2 


* 


5.8 


7.2 


5.4 


0.4 


* 


8.0 


22.2 - 


D.8 


8.0 


11.6 


Coronal polishing 


2.2 


0.0 


--0.8 


2.6 


4.6 


2.8 


* 


6.2 


5.8 


2.4 


8.0 


* 


Polishing amalgam 


10.4 


0.2 


1.0 


1.8 


5.6 


0.0 


* 


2.0 


8.8 


- * 


5.8 


8.4 


restoration 


















<* 








Study cast 


6.3 


0.0 


5.3 


6.1 


12.0 


7.5 


11.8 


2.1 


8.4 


0.0 


8.0 


6.0 



'* Indicates test not given at that institution. 
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better than the overall figures would suggest. This possibility is worth 
investigating further. 

Observer s* iudgmewts of examinee perforD <» >ce. Table 15 summarizes 
the observers' ov v-iLl judgments of the examinees' perf orc/ance. The total 
number of examinees listed for each test l3 less than the coirf:^-?^on-.Uuf, number 
in Tables 11 and 13 because in some cases one or both observer^ neg^Iexx^^^ t^- 
check either "a^dec^xat«"-o The observer's judgment for the 

performance test in taking vi,tal signs is a combination of three separate 
judgments (for pulse rate, respiration rate, and blood pressure). ^ judgmsnr: 
of "adequate" for this test .means that the examines was judged adequate on all 
three parts of the t^st. 

The observers' overall judgments disagreed about a substantial number 
of examinees on some of the tests, particularly the surgical scrub and the 
alginate Impression. Tliese disagreements can probably be attributed to two 
factors: (1) unreliability in the observation process, and (2) differences 
between observers in the standards by which they judged examinee performance* 
That is, the tobservers may have disagreed about what the examinee did, or 
they may have agreed about what the examinee did but disagreed as to whether 
or not it represented adequate performance. Unreliability of the observation 
process would be reflected in the examinees' scores, while differences in 
observers' standards generally would not* Since inter-observer reliability 
was a problem in the surgical scrub, it is not surprising to find a substantial 
number of disagreements in the observers' judgments on that test* The inter- 
observer reliability of the scores on the alginate impression test was not 
particularly poor* However, some of the specific checkpoints that discrimi- 
nated best between ad quate and inadequate performers were also the least 
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Table 15. Observers' Judgments of Examinees' Performance 



. 














Performance Test 






Observers' Judgements 








Both 
"adequate" 


One "adequate," 
one "inadequate" 


Both 
"inadequate 


Asslstlp.-* with restorations 


49 


(72Z) 


5 


( ?%) 


14. (21%) 


Surgical scrub 


26 


(23%) 


26 


(23%) 


59 


(53%) 


Oral hygiene Instruction 


43 


(41^) 


15 


(1A%) 


47 


(45%) 


Vital signs 


36 


(37Z) 


11 


(11%) 


50 


(52%) 


Radiography 


55 


(63%) 


10 


(1.1%) 


' 22 


(25%) 


Alginate Impression 


49 


(69%) 


15 


(21%) 


7 


(10%) 


Periodontal dressing 


27 


(61%)' 


5 


(11%) 


12 


(27%) 


Coronal polishing 


21 


(64%) 


1 


( 3%) 


■11 


(33%) 


Polishing amalgam restoration 


28 


(54%) 


6 


(12%) 


18 


(35%) 


Study cast 


49 


(52%) 


11 


(12%) 


35 


(37%) 
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reliable. In other words, some aspects of the procedure that the observers 
seem to have considered roost important were tho8e_ in jihich -they vere most 

abbut'what the examinee had done. All of these check- 
points were in the evaluation of the finished impression. Possibly the 
Inter-observer reliability of these checkpoints could be improved by giving 
the observers photographs of ■ acceptable and unacceptable alginate impressions 
to use as examples, as is done in the radiography test. According to the 
observers' overall judgments, the examinees as a group seem to have performed 
best in assisting with restorations and in taking the alginate impression; 
worst in the surgical scrub. These judgments are consistent with the 
examinees' scores on the tests as shown in Table 11. 

Doveloprnent of the written tests 

Content specifications. The content specifications for the written 
tests were determined at the same time as those for the performance tests. 
First Che Advisory Committee reviewed the task analysis document .and grouped 
the tasks into major categories. These categories were similar but not 
identical to- those ot the task analysis document. After grouping the tasks 
into categories., the committee suggested the degree of emphasis to be 
placed on eacii category. The corunittee also recommended that item writers 
'oniiider the cognitive ability levels of Bloom's Taxonomy . * 



Benjamin S. Bloom, Taxonomy of Educational Objectives; Cognitive Domain . 
New York, David McKay, 1956. 
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At the first meeting of the Working Committee the Written Test Sub- 
committee reviewed the task statements in the task analysis document, making 
changes they deemed appropriate. They reviewed and revised the content 
specifications suggested by the Advisory Committee and classified each task 
into the appropriate content category. The subcommittee reviewed several 
approaches to the abilities dimension and decided to ask the persons writing 
the^ test items (questions) to write at the "knowledge" and "application" 
levels of Bloom's T axonomy * The members of the subcommittee then wrote a 
number: ,of prototype questions illustrating each task and ability level. 
These prototype questions were included in the materials that ETS sent to 
the item writers. 

Writing of test items . The tests in the Proficiency Examination are 
criterion-referenced tests, which are, by definition, "deliberately con- 
structed to yield measurements that are interpretable in terms of specified 
performance standards."* This fact has some important implications for the 
writing of .test items, which have been expressed as follows in another 
well-known article on the subject: 

"The basic difference between item construction in norm-referenced 
and criterion-referenced frameworks is a matter of "set" on the part 
of the item writer. . . - 

Most important, when a writer constructs items for a norm-referenced 
test, he wants variability and, as a consequence, makes all sorts of 
concessions, sometimes subtle, sometimes obvious, to promote variant 
scores. He disdains items which are "too easy" or "too hard»" He 
tries to increase the allure of wrong answer options. . . 



Robert Glaser, "A criterion-referenced test," in W. James Popham (ed.) 
Criterion-Referenced Measurement . Englewood Cliffs, N.J.: Educational 
Technology Publications, 1971. 
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The criterion-referenced item writer is guided by another gbal. 
His chief rule is to make. sure the item is an accurate reflection of 
the criterion behavior. Difficult or easy, discriminating or indis- 
criminate, the important thing is to make the item represent the class 
of behaviors delimited by the criterion. Those "^who write criterion- 
referenced items are usually far more attientlve to defining the domain 
of relevant test responses and the situations in which they should be 
required. This rather fundamental difference in "set" on the part of 
criterion-referenced and norm-referenced item writers can clearly 
contribute to differences in the resulting items."* 

ETS selected 22 persons to write the multiple-choice items, selecting from 

the same llsta used to select persons for the Working Committee, augmented by 

additional suggest ions from members of the Advisory Committee and the Working 

Comciittee. ETS contracted with each item writer for twenty items in a 

specified content area, so that the number of items written in each content 

area was approximately twice the number needed. (The purpose of this 

"overage" was to allow for the deletion of items that were redundant with 

each other or in some way inappropriate for the examination.) ETS provided 

each item writer with four types of information: / 

1. Task statements for the dental auxiliary tasks about which 
the item writer was to write questions; 

2. A description of the desired cognitive ability levels, 
"knowledge" and "application," and examples of questions 
written at these cognitive ability levels; 

3. A set of instructions for writing good test questions, 
including examples of questions illustrating these 
Instructions ; 

4. Descriptions of the item formats to be used in the 
Proficiency Examination. 



W. James Popham and T. R. Husek, "implications of Criterion-referenced 
measurement," J ournal of Edu catio nal Measurement , 1969, Vol. b, no. 
pp. 1-9. 




The item writers sent their test. items to ETS, where ETS test devel- 
opers edited them for style, grammar, and format and reviewed them for certain 
types of flaws commonly found in multiple'^choice items* 

Review by subcommittee > The Written Test Subcommittee met in 
Alexandria, Virginia on August 18, 19, and 20, 1978. Before the meeting, 
each subcommittee member had received a copy of all the items to be reviewed* 
The subcommittee members were asked to answer each item (without benefit of 
the answer key) and to rate each item on three separate criteria; technical 
quality, significance of content, and appropriateness for the assigned cate- 
gory in the test specifications. In addition, the subcommittee members were 
to offer any suggestions they had for improving the itetp. The instructions 
to the subcommittee members emphasized two major themes they were to keep in 
mind when reviewing each test item: 

1. "Is it lob-related ?" 

2. "Is it appropriate for an entry-level dental auxiliary?" 

The items,. with the subcommittee members' answers, ratings, and comments 
were mailed to ETS, where they were collated and duplicated so that each 
subcommittee member could see all the members' responses to an item at once. 
At the subcommittee meeting the members reviewed their responses and ratings 
of each item* They then decided, for each Item, to accept it witjjout ^ 
revision, to revise it, or to delete it from the item pool. Of 430 items 
submitted, 315 survived this review process* In one or two of the specified 
content categories the number of items that survived the review process was 
less than the number called for in the specifications* Members of the sub- 
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conunittee volunteered to write additional items in these categories to meet 
the specifications. 

Review by committees. Before the written test items. were assembled into 
booklet form for pretesting, they were reviewed by the full Working Committee 
and the Advisory Committee. The Working Committee reviewed each written test . 
item Individually Lo make sure that its technical content was correct, impor- 
tant, and appropriate for the examination and that its language was' correct 
and appropriate. Several items were revised, some were deleted. The Working 
Committee also identified a subset of items in diagnosjtic aids, therapeutics, 
and patient education as appropriate for dental hygienists but not for dental 
assistants. The Advisory Committee also reviewed the written test items, and " 
suggest.ed revisions to several of the items. 

Pretesting and revision of the written test icems . The- written 'tests 
Were pretested on a total of 24 examinees at the five test sites. The pretest 
data from the written tests were analyzed separately for each olf, the five' 
competency areas. The examinees were divided into two groups on the basis' 
of thdlr scores. Questions missed frequervtly by members of the high-scoring 
group were Identified for special review, to detect ambiguities of phrasing,^ 
"correct" answers that might under some circumstances 1»e inc^orrect, and * 
"incorrect" answers thar might under some circumstances be correct. Although 
this type of analysis is'^c^jionly u^ed in constructing norm-referenced tests, 
it is appropriate for criterion-referenced tests as well. Thfe difference lies 
in the decisions made as a result of the analysis: ♦ 

"Item analysis procedures have traditionally been used with norm- 
referenced tests to identify those items that were not properly dis- • " 
criminating among Individuals taking the test. For instance, in an 
achievement test .an unsatisfactory item would be one which could not 
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properly alscriminate between the more and less knowledgeable learners 
(as reflected by total test performance)* Non**dl8crlmlnat Ing Items are 
usually* those which are (a) too easy, (b) too hard, and/or (c) ambiguous. 

For trriterion-ref erenced tests the use of discrimination indices must 
be modified. An item which doesn't discriminate need not be eliminated. • 

However, negatively • iiscriminating items are treated exactly the sanje 
way in a criterion-referenced approach as they are in a norm-;ref erenced / 
• approach. An item which discriminates negatively is one which, in an 
instructional context, is answered correctly more otten by. the less 
knowledgeable than by the more knowledgeable students. When one dis- 
covers a negative discriminator in his pool of criteri'jn-ref erenced 
itenvs, he should be suspicious of it *and after more careful analysis can 
usually detect flaws in such an item.*'* 

The Wo^rking Committee reviewed the test items and the. pretest data at 

f 

the committee's third meeting, paying specJ>iNpt tent ion to the items identi- 
*^fied as possibly defective on the basis of the pretest data. Committ<?e 
members rewrote several of these items.. Pretesting had also indicated that 
the written tests req.uired less time than had been anticipated, and the 
committee agreed to keep a larger;-than--anticipated number of questions in 
the written tests and to shorten /the testing time. 

Pilot testinj^of the written tests . A tot^il of 115 examinees — 58 
dental assistants and 57 dental hygienists took the written tests during 
the pilot testing. Because some of the test questions had been idenrlfied 
as appropriate for dental assistants but not for dental hygienists^ ETS 
printed two different examination booklets. The tests in chairside asgisting 
and in laboratory procedures were the same in both booklets. » For the tests 
in patient education, diagnostic aids, and therapeutics, the hygienists' 
. ^booklet contained the longer versions and the assistants' booklet contained 



*W. James Popham and T. R. Husek, op. clt. 
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the shorter versions. The time limits fpj' these three tests were adjusted 
accordingly. The written tests were scored without a penalty for incorrect 
guessing. The time limits were ample; all examinees completed all five 
written tests. The scores are summarized in Table 16, where they are 
expressed in percent-coi rect terms. Although the tests were constructed to 
measurp er:ry-level knowledge and skills and the pilot test examinees were 
all Certified Dental Assistants, Registered Dental Assistants, or Registered 
Kcntal Hygienists, their scores did not tend to cluster in the highest score 
brackets on most of the tests. 

Rplia bility of the written tests . Reliability is the tendency of a 
Lost to give consistent scores — scores that change very little even when 
tl.. particu.ar set of questions on the test is replaced by another set of 
questions testing the same topics. The extent to which the scores could be 
expected to change is indicated by a statistic called the "standard error of 
Hfasurenont'^ This statistic represents the difference between the score 
an exatnint^c actualLy nade and the average of all the scores the examinee 
Would make if he or she could somehow take all possible versions of the 
tfst^ Tho standard error of measurement is the estimated average size of 
t!u»se di t f crtMicos for all examinees taking the test* The smallei the 
standard error ot measurenent is, in relation to the length of the test, 

novo ri»llable the test scores are* Table 17 shows the standard error 
of neasurernent for each of the written tests, based on the pilot tes*: 
data* The number *'i,4'* in the first line of Table 17 can be interpreted 
as follows: Suppose ETS »ere to construct a large number of written tests 
in chairside assisting, each consisting of 64 questions on the specified 
t'^plrs* Suppose the pilot test examinees had taken all of these tests and 
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Table 16* Distributions or Scores on Written Tests* 



Test 
NuMfaer 


of 


Chairaide 

Assisting 


Pat lent 
Education 


Diagnostic 
Aids 


Therapeutics 


Laboratory 
ProcedurM 


questions 


Oh 


26» 


HO** 


49* 


60** 


15« 


36** 


28 


Percent 


correct 


















91 . 


100 


n 
u 


0 


6 (IIS) 


0 


3 ( 55) 


1 ( 25) 


1 ( 25) 


0 


81 - 


90 


7 f \ 

/ ( 6%; 


13 (22S) 


}b (61%) 


17 (29S) 


40 (705) 


4 ( 75) 


16 /285) 


17 (155) 


71 - 


80 




17 (295) 


16 (28«) 


22 (38%) 


11 (195) 


16 (285) 


2:} (405) 


35 (305) 


61 - 


70 




23 (UOfi) 


0 


13 (265) 


3 ( 55) 


14 (245) 


14 (255) 


47 (415) 


51 - 


60 


zU ( 1751) 


3 I b%) 


0 


4 ( 75) 








12 (105) 


41 . 


SO 


^ ( k'i) 


2 ( 315) 


0 


0 


0 


8 (145) 


1 ( 25) 


3 ( 35) 


31 . 


40 


2 ( 2%) 


0 


0 ' 


0 


0 


1 ( 25) 


0 


1 ( 15) 


0 - 


30 


0 


0 


0 


0 


n 




n 
U 


0 


Number of 


















examinees 


115 




57 


58 


57 


58 


57 


lis 


Average 




















percent 




















correct 




66.0 


71.2 


84.4 


74.8 


83.4 


64. B 


74.5 


68.9 


Standard 


















deviat ion 


















{% correct) 


9,2 


a.b 


^.6 


9.1 


6.0 


14.2 


8.3 


9.4 



•Assistants* version 
••Hyqienista' version 



Note: Because the hyqienists* version of the tests in patient education, diaqnostic aids, and therapeutics contained several 
questions not on the assistants' version; scores on the two versions are not romparable. 
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Table 17. Standard Error of Measurement of the Written Tests. 



Number of Standard error 

'^^^^ questions of measurement 



Chairside assisting 64 

Patient education 

assistants' version 26 

hygienists' version 40 



•71 



3.4 



K9 
2.0 



Diagnostic aids 

assistants' version 49 2.7 

hygienists' version 60 2.6 

Therapeutics 

assistants' version 15 1.7 

hygienists' version 36 2.3 

Laboratory procedures 28 2.1 



ETS had computed each examinee's average score over all the tests in chair- 
side assisting. A typical examinee could expect to find a difference of 3.4 
questions correct (equal to 5.3 percentage points) between his or her score 
on any one of these tests and his or her long-tern average score. 

Estimated reading difficulty of the written tests . ETS staff computed 
the estimated reading difficulty of each of the written tests by applying the 
"SMOG" index to the tests in their finished form.* The SMOG index gives the 
estimated reading difficulty of a written passage in grade-level terms. That 
is, if a passage has a difficulty level of 10, it can be read and understood 
fylly by a person who reads as well as an average tenth-grade student but not 
by a person who reads only as well as an average ninth-grade student. The 
estimated reading difficulty levels^ of the written tests ranged from 10.3 to 
10.8. Therefore, an examinee who reads as well as an average llth-grade 
studenC should have no r'ifficulty in reading the test. 

Estimated reading difficulty is a function of two factors: sentence 
length and frequency of difficult or unfamiliar words. (The SMOG index uses 
the number of words of three ot more syllables as an indicator of the number 
of dlfficoilt words.) Inspection of the written tests revealed that it would 
be possible to reduce their estimated reading difficulty (and presumably their 
actual reading difficulty as well) to some degree by breaking up some long 
sentences into shorter ones. However, many of the longer words are technical 
terms. Any rephrasing of the questions would have to be reviewed for technical 
correctness. 



*G. H. McLaughlin: "SMOG grading: A new readability formula," Journal of 
Reading,, 1969, Vol. 12 pp. 639-646. 
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Recommendations for Pass/Fall Cutoff Scores 
If a test consisted of only a single question or checkpoint, choosing 
a pass/fail cutoff scoria would not be a problem. Any examinee who answered 
or performed correctly would pass; any examinee who answered or performed in- 
correctly would fail. But when the results of many questions or checkpoints 
are combined into a single score, someone must decide how high a score to 
require for an examinee to pass the test. Unfortunately, even the best 
examinees — persons who are undoubtedly proficient In the subject of the 
test — rarely get perfect scores. Therefore, the process of choosing a 
pass/fail cutoff score requires some form of judgment. 

The judgments Involved In choosing a pass/fail cutoff score must be 
made by persons who are qualified to make them, and they must be made in a 
way that is meaningful and realistic. It is possible simply to have the judges 
examine the test and choose a particular score as the pass/fail cutoff. How- 
over, experience has shown that this simple method often results in cutoff 
scores that are clearly unrealistic, in that large members of competent 
examinees would fail the test if the resulting cutoff score were used. As 
a result, several alternative methods have been proposed. These methods 
can generally be categorized in^o two groups. One group consists of 
methods based on judgments as to the way a hypothetical •'minimally qualified" 
examinee wuuld perform on the test. This "minimally qualified" examinee is 
the borderline case — the examinee whose proficiency just barely meets the 
standard. These methods do not require the actual identification of any 
such examinees. Instead, the judges are asked to think of real or imaginary 
persons who fit the defi lition and make an Informed guess — a conjecture — 
as to how these persons would perform on the test. Therefore, these methods 
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can be called "conjectural methods The second gMup consists of methods 
based on Judgments of the proficiency of real examinees whose test scores 
are known. These methods require the collection of data in the form of the 
examinees' test scores and the judges' evaluation of the examinees' profi- 
ciency* Because these methods are based on observed data, they can be 
called "empirical methods." 

Making informed guesses about the test performance of a hypothetical 
"minimally qualified" examinee is a somewhat abstract and unfamiliar task. 
Making judgments about the proficiency of real persons is a fairly concrete 
and familiar task, particularly for instructors and supervisors of perspns 
in the job for which the examinee is being tested (although individual 
judges may differ in the standards they apply). Therefore, the empirical 
methods are generally preferable to the conjectural methods — if the 
necessary data can be obtained. This is sometimes a difficult requirement, 
since the judges must have enough information about the examinees to make 
meaningful judgments. Also, the judges' evaluations must be independent of 
the examinees' test scores, at least to the extent that the judges do not 
know the examinees' scores at the time they make the judgments. Fortunately, 
this type of data was obtainable from the pilot testing of the performance 
tests (except for charting from dictation), since each examinee's performance 
was observed by two observers who were qualified in the tasks involved in 
the test. The observers made their judgments after observing the examinee's 
performance, by checking one of two spaces, one marked "adequate" and the 
other "inadequate." Since the observers in the pilot testing did not know 
tho scoring formulas, their judgments were made independently of the 
examinees' scores. (In fact, the final determination of the scoring formulas 
had not been made at the time of the pilot testing.) However, for the written 
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test no such Judgments were obtainable. Each of the written • tests measured 
a broad range of knowledge. It was not possible to obtain a Judgment of 
the adequacy of each examinee's knowledge from a person qualified to make 
such a Judgment. Therefore, ETS chose an empirical method for the perform- 
ance tests (except for charting from dictation, in which the examinee's 
chart was not directly .observed) and a conjectural method for the written 
tostt. and the performance test in charting from dictation. 

.^.^it^ li^jscQXgs^ f Q-f the performance tests . 

The method ETS used to recommend cutoff scores for the performance 
tests Is an application of statistical decision theory. In this method, the • 
examinees' test scores and the observers' judgments are used to estimate the 
rol.Jtionship between an examinee's test score and the probability that an 
observer would judge the examinee's peJforraance to be adequate. 

Table 18 shows the distributions of perfor.aance test scores, classified 
according'. Lo the observer's overall judgment of the examinee's performance. 
Each checklist is counted separately in this table, so that an examinee who 
was judged adequate by one observer and inadequate by the other observer would 
be counted once in each column. The scores for examinees Judged adequate are, 
of course, higher than those for examinees judged inadequate, but there is a 
substantial overlap. There are at least two factors that could have caused 
this overlap in the distributions. First, the observers undoubtedly varied 
somewhat In the standards by which they Judged the examinees' performance* 
Second, the observers' criteria for judging examinees' performance may have 
differed from the criteria embodied in the scoring formulas* The estimates 
of the probability that an examinee with a given test score would be judged 
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Table 18. Distributions of Performance Test Scores, Classified by Observer's Judgment. 

(A : "adequate;" I = "inadequate") 



Performance 
lest 



Assist iriq with 
Restorat ions 



Surgical 
Scrub 



Oral 
Hygiene 
Instruction 



Vital 
Signs 



Radiography 



Alginate 
Impression 



Obspr ver 's 



judqment 


A 


I 


A 


I 


A 


I 


A 


I 


A 


I 


A 


I 


lest Score 


























96 - 


ion 


68 




9 


0 


5 


0 


65 


9 


0 


0 


21 


0 


91 - 


95 


36 


11 


IS 


0 


3 


1 


3 


1 


5 


0 


20 


1 


86 - 


9U 


6 


16 


11 


9 


6 


2 


13 


37 


7 


0 


45 


7 


81 - 


8S 


0 


4 


11 


4 


18 


1 


3 


22 


17 


0 


19 


7 


76 - 


80 


0 


0 


12 


6 


17 


3 


0 


9 


23 


2 


6 


5 


71 - 


7S 


0 


0 


2 


4 


15 


10 


2 


26 


24 


5 


4 


5 


66 - 


70 


0 


0 


3 


1 


20 


9 


3 


14 


21 


9 


0 


1 


61 - 


65 


0 


0 


1 


6 


7 


8 


0 


. 2 


14 


6 


0 


1 


56 - 


60 


0 


0 


2 


3 


6 


17 


0 


1 


9 


io 


0 


0 


51 - 


55 


0 


0 


4 


20 


2 


9 


0 


0 


4 


4 


0 


1 


46 - 


50 


0 


0 


1 


9 


6 


8 


0 


0 


3 


4 


0 


1 


41 - 


45 


0 


0 


2 


33 


2 


4 


0 


0 


2 


9 


0 


0 


36 - 


40 


0 


0 


4 


20 


2 


12 


0 


• 0 


0 


0 


0 


0 


31 - 


55 


0 


0 


1 


9 


0 


7 


0 


0 


0 


0 


0 


0 


0 - 


30 


0 


0 


0 


23 


0 


3 


0 


0 


0 


0 


0 


0 































Nuniber of 
checklists 



110 



36 



78 



147 



109 



94 



89 121 



129 



57 



115 



29 
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Table IB. (continued) 



Performance 
lest 

Observer's 
judgment 

Test Score 
96 - 100 

91 - 95 

86-90 

81 - as 

76 - 80 

71 - 7^ 

66 - 70 

61 - 6S 

^6 - 60 

51 - 55 

liS - 50 

A1 - 45 

36 - 40 

51 - 55 

0-50 

Numbelf of 
checklists 



Penoilontal 
Dressing 



A I 

9 0 
10 14 



Coronal 
Pol ishing 



15 

11 

15 
1 
1 
L 
0 
1 
3 

0 

0 

0 

0 



0 
7 
5 
2 

0 
1 
1 
3 
1 
1 
0 
1 



13 
6 
7 
7 

9 
5 

2 
3 
1 
1 

0 
0 
0 
0 
0 



1 

0 

1 

4 

7 
3 

3 
1 
4 
2 

0 
0 
0 
0 
0 



Polishing 
Amalgain 
Restoration 



I 



Study 
Cast 



15 

19 
4 

17 
9 
1 
1 
1 

0 

0 

0 

0 

0 

0 

0 



0 
0 

1 

8 
15 
7 
7 
1 
0 
0 
0 
2 
1 
0 
0 



9 
9 
12 
9 

10 
27 
11 
10 

8 

3 

3 



0 
2 
1 



0 
0 
1 
2 

6 

7 
5 
6 
7 

11 

11 



0 15 



3 
4 

4 



62 51 



54 26 



77 



67 42 



114 82 
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adequate are shovm in Table 19. Obviously, the relationship between test 
score and probability of being judged adequate at the tasks tested varies 
greatly from test to test* 

To use information of this type to choose a pass/fail cutoff, one must 
first specify the relative seriousness of the two types of decision errors 
that might be made: (1) passing an inadequate performer, and (2) failing an 
adequate performer* Statistical decision theory tells us to choose the 
pass/fail cutoff that will minimize the total harm from the decision 'errors 
that will be made* If the two types of errors are equally serious, the 
decision maker should simply try to minimize the number of decision errors* 
If an examinee's test score indicates that the examinee would be more likely 
to be judged adequate than inadequate, the examinee should pass* The cutoff 
should be the score at which the examinee is as likely to be judged adequate 
as inadequate — the score that corresponds to.^a •bO probability of being 
judged adequate* 

What If the two types of errors are not equally serious? For example, 
suppose it Is twice as harmful to pass an inadequate performer as to fail an 
adequate performer? In this case, the cutoff should be the score at which 
the examinee is t wice as likely to be judged adequate as inadequate* This 
is the score that corresponds to a #67 probability of being judged adequate* 
Table 20 shows the recommended cutoff scores for several such ratios of 
importance of avoiding the two error types (which decision theorists often 
refer to as the "ratio of regrets")* At the final Working Committee meeting, 
the committee members were asked to Judge the relative importance of- the 
two types of decision eriors for each of the performance tests* After 
discussion, the committee agreed on a ratio of 2:1 (indicating that the 



Table 19. Estimated Probability Thai an Uaminee's Performance Would be Judged Adequate. 



Oral 



last. 'iTr^ T""' . ''''' '^^"^"^^^ Periodontal Coronal ^sl^^ Study 

Test. Restorations Scrub Instruction Signs Radiography Impression Dressing Polishing Restoration C-st 



Score: ion l.oo 1.on i.on 



^•00 1-00 ^.00 1.00 1.00 1.00 1.00 



9^ -9^1 .92 .96 ' .81 1.00 



•99 .96 .95 1.00 .98 



^° •''^ -80 ^-^^ ' .91 .94 .94 



8^ -10 .^1 .86 .33 .98 

80 -00 .65 .81 .18 .94 



•82 .80 .80 .75 .85 

•62 .67 .65 .57 .80 

•^6 .55 . .55" .33 .76 

•'0 •'JZ .51 .U .74 

•10 ^37 .48 .13 .65 

•00 .35 .42 .08 .50 

•00 .35 .28 .01 .33 

•00 .32 .12 .00 ,25 

.22 .00 .28 .00 .25 .03 .00 .18 



75 .00 .59 .74 .11 .85 

70 •OO .54 .65 .09 .76 

65 -00 .48 .57 .06 .61 

60 .00 .36 .48 .05 .53 

55 -00^ .27 .35 .01 .43 

50 .00 .16 • .28 .00 .39 



*0 .00 .10 .16 .00 .16 



•00 .11 .00 .00 .17 



ERIC 



Table 20* Racommended Pass/Fail Cutoff Scores for the Performance Tests and Resulting Failure Rates for Pilot ^%t Examinees^ 
^ (Scores are Expressed aa Percent of Posaibie Score.) 



Ratio of Regrets • 

Assiatinq with restorations 

Surgical scrub 

Oral hygiene instruction 

Vital signs 

Radiography 

A Iq ina t e imp res s i on 

Periodontal . 

Coronal polishinq 



1:2 



1:1 



2:1 



3j1 



4:1 



cutoff failure cutoff failure cutoff failure cutoff failure cutoff failure 



score 
88 

y* 

59 
83 
<t8 
72 
53 
58 



Polishing amalgam restoration 74 



rate 
15% 



34* 

8% 
4% 
B% 

^o% 

19S 



study cast 



55 



score 
91 

67 

6S 

88 

59 

77 

73 

65 

79 

60 



rate 



43S 



12!S 
22% 
15% 
39% 
37% 



score 
92 

91 

70 

92 

67 

82 

81 

.82 

82 

66 



rate 
26% 

77% 

57% 

64% 

44% 

19% 

55% 

60% 

49% 



score 
93 

87 

74 

94 

70 

84 

84 

84 

85 

73 



rate 
34% 

87% 

67%:^ 



30% 
63% 
64% 
63% 
66% 



score 
94 

90 

76 

95 

72 

85 

86 

85 

88 

80 



rate 
45X 

91» 

73% 

70% 

. 63% 

36% 

65% 

67% 

70% 

62% 



•Relative importance of ayoidinq the two types of possible decision errors, e.g., "4:1" means "if it is four times 
as harmful to pass an inadequate performer as to fail an adequate performer." 
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former type of error is twice as serious as the latter) for the performance 
tests in chairside assisting, diagnostic aids, and dental therapeutics, /and 
a ratio of 1:1 (indicating that the two types of errors are equally serious) / 
for the performance- tests in patient education and laboratory procedures. 

t 

One member of the committee dissented, maintaining that ^,t he two types of 
errors were equally seriou« for all the performance tests. 

iL"A9 f^ sco res for the wrlti^en tests 

Because of the difficulty of obtaining direct judgments of examinees' 
proficiency In the knowledge tested in the written tests, ETS decided to use 
a conjectural niothod for recommending cutoff scores on the written tests. 
Throe basic methods of this type have been proposed; they are usually 
Identified by the names of the authors who first proposed them. "Ebel's 
method"* calls for the Judges to classify the test questions according to 
thoir difficulty and importance and then to judge what proportion of the 
questions in each categorpa minimally competent examinee wo^ild answer ' 
<:orrectly. ''Angoff's method"** calls for the judges to state the probability 
that a minimnlly competent examinee would answer each question correctly (or, 
alternatively, what, percentage of a large group of such examinees would answer 



Robeit L. Ebel, Ess ential s of Educ ational Measurement; Englewood 
Cliffs, New Jersey; Prentice-Hall ,"l972, pp. 492-49A. 

William Angoff, "Scales, norms, and equivalent scores," in 

R. L. Thorndike (ed.), j^c ationa 1 Mo as u r emcnt ; Washington DC ; 

American Council on Education, 1971, p. 531. 
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t 

each question correctly) • *'Nodelsky 's method*'* calls for the Judges to 

determine which ^rong answers the minimally competent examinee would be able 

« * 

to eliminate from consideration. 

Only a small amount of research has been done to attempt to determine 
the probable effects of using one or ano.ther of the conjectural methods* One 
study** compared Ebel's method with Nede^sky's and found that Ebcl's rjethod 
resulted in systematically higher cutoff scores. Another study*** compared 
Nedelsky's method with an empirical method (assuming equal importance of the 
two types of decision errors) and found no consistent tendency of the Ncdolsky 
method to produce cutoff scores that were either higher or lower than those 
produced by/ the empirical method. 

ETS tried all three conjectural methods at the fourth Working Committee 
meeting. ^ The committee members served as judges, although not all the members 
completed all three methods. They made their Judgments without any informa- 
tion about the performance of the pilot test examinees. Table 21 presents the 
cutoff scores as compuVed^ f rom the judgments of each committee member under 
each method, and the resulting f^vi^lure rates in the pilot test examinee group. 
Judges 1, 2, 3, and 4 were dental assistants; they judged the assistant5>' 



Leo Nedelsky, ''Absolute grading standards for objective tests/* 
Kducational a n d Psycholop.lcal Measurement , 1954, Vol. 14 no. 1, 
pp. 3-19. 

**Bat^bara J. Andrew and James T. Hecht, A preliminary investigation of 
two procedures for setting examination standards. E ducational and 
Psychological Mea su reme nt, 1976. Vol. 36 No. 1, pp. 45-50. 

**StepHen L. Koffler, A comparison of approaches for setting proficiency 
standards* Unpubllsho'^ manuscript, frenton. New Jersey; New Jersey 
Department of Education, 1979. 



fibie 21. Written test Cutoff Scores by ThUio DiXliiriMtTyelhodo arid Resultifnj failure Rates for 

Oiairfiide AriSittt loq f it»nt tducaliofi Patient tducation 

• v;.- ' ^ Iti»ftis AttBistanttt: 26 ItemR Hyqienists: 40 Items 



Pilot Test Examinees 

Diagnostic Aids 
Assistants 2 49 lies 







(6el 


AfMinf f 




i f\m 1 

V OC i 




Nedc 1 sk y 


ttM?! 


Angof f 


Nedelsky 


Cbel 


Angof f 


Nedsltky 




Cutoff wort* 


w ■ 

"yir« 




41 ' 


21 
78^ 


I*) 

4fl% 


17 
17% 








40 
71% 


34 
26% 


37 


judge 2 


Cutoff scoff 
^Bilure fdtt^ 


a? 


» •• # 


42 


21 


21 


23 


* — 






36 
3BSi 


34 
26% 

« 


38 
52% 


jMdqe 5 


Cuta^f s'?cre 
failuri^ rdtf^i 


SI 






21 


16 




/ " 






38 


34 
26% 


1 — 


JlHJije 4 


Cutoff ururr 
f fiilijro r^fttt* 












16 










MM 




Cutoff Ktore 

fcilurt* rati* 
« 










22 
9^% 


7% 


55 
?0% 


^3 
28% 


24 

0% 




42 
86% 


33 

2n 


Judge 6 


Cutoff icuri* 
i allure rote ' 


60 
109% 








22 
95% 


20 
66% 


" 37 
90% 


34 
42% 


30 
4% 




4T 
74% 


38 


Jbdye' 7 


Cutoff ttcor^ 
faiJI^ure rot*' 






84% 




22 
9^% 


19 
48% 


33 
28% 


33 
28% 


31 
9% 




41 

74'*' 


36 




Cutoff f;<^ore 






41 
36% 






21 
78% 






32 
16% 






26X 
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Table 21 (continued) 



'Judqe 1 Cutoff score 
Failure rate 

Judc, 2 Cutoff score 
Failure rate 

Judge 3 Cutoff score 
Failure rate 

Judge ,4 Cutoff score 
Failure rate 



/ 



I 

Diagnostic Aids 
Hyqienists; 60 Itema 



/ 



Tfl^rapeutics 
Assistants: 13 'items 



Therapeutics 
Hyqienists: 36 I terns 



Laboratory Procedure^ 
28 Items 



Ibel Anqoff Nedelsky Ebel Anqoff Nedelsky Cbel Anqoff Nedelsky Ebel Angoff Nedelaky 



12 
60% 

11 
(A% 

12 



11 
11 

m 

80% 



10 
40% 

14 
98% 



40J 



23 
83% 

20 
40% 

22 
69% 



20 
40% 

20 
40% 

29 
40% 



19 
30X 

20 
40X 



Judqe b Cutoff score 32 41 

Failure rate 7U% 65% 2% 

Judge 6 Cutoff score 38 32 48 

Failure rate 100% 63% 23% 

Judge 7 Cutoff acorft' 30 3() 4a 

Failure rattj 30% 30% 7% 

Judge 8 Cutoff score — — 41 

Failure rate 2% 



13 
91% 

13 
91% 

12 
80% 



11 



12 
80% 

11 
64% 

13 
80% 



32 
98% 

35 

loor, 

33 
98% 



31 



31 
91! 

31 



27 
42% 

27 
.'52% 

23 
20% 

26 
30% 



25 
96% 

25 
96% 

23 
83% 



24 
87% 

Z4 
87% 

23 
83% 



22 
69X 

18 
ITS 

19 
30X 

21 
57% 
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version of the tests In patient education, diagnostic aids, and therapeutics. 
Judges 5, 6, and 7, who were dental hygienists, and Judge 8, who was a dentist, 
all judged the hygienijsts' version. 

In general, Ebel's method prod'^uced higher cutoff scores than Angoff's 
or Nedelsky's . However, all three methods produced at least one or two 
cutoff scores that seem unrealistically high, considering that the examinees 
wert; practicing dental assistants and dental hygieni^ts holding current 
credentials. One possible reason for the very high cutoff scores might be 
the fact that the judges were the same persons who had developed the test. 
As an Informal test of this hypothesis, an ETS staff member instructed the 
two dentists and the two dental hyglenists on the Advisory Committee in the 
Nedelsky procedure and asked thera to apply it to one of the shorter written 
tests, "'e resulting cutoff scores were substantially lower than those 
conputud from the judgments made by the Working Conunlttee members. ETS 
therefore recommends that, before the written tests are used for creden- 
tlaHng purposes, another set of judgments should be collected, this time 
from persons who were not involved in the development of the examination.. 
If cutoff scores must be chosen without the benefit of another set of 
jitdgnents, ETJ recommends that the cutoff score for each test be the median 
ot the cutoff scores from the Nedelsky method. These cutoff scores are 
shown in Table 22,. In percent-correct terms, the cutoff scores for the 
written tests range from 64 percent to 75 percent correct. The corres- 
ponding failure rates-in the full pilot test examinee group (assistants and 
hyglenists combined) range from 20 percent to 47 percent. These failure 
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Table 22. Suggested Cutoff Scores for the Written Tests 
(pending further Investigation)* 



Test No. of Itetis Cutoff sco re 



Chairslde assisting 




64 




4.U 


Patient education: 


assistants 


26 




19 




hyglenlsts 


- '40 




30 


Diagnostic alds_, ; - 


assistants 


49 




36 




hyglenlsts 


60 




42 


Therapeutics : 


assistants 


i5 




11 




hyglenlsts 


36 




26 


Laboratory procedures 


28 




19 



9 



86 



rates seen somewhat high, but^they are not generally higher than the propor- 
tions of examinee* judged inadequate on the performance tests, and therefore 
do not seem wildly unrealistic. 

The performance test in charting from dictation does not involve direct 
observation of the examinees' charts. Instead, the examinee uses his/her 
chart to answer twelve multiple-choice questions about specif ic -fef.tures 
that were to be cl.arted. Members of the Working Committee were asked to 
apply the Nedelsky Method to this test. Six members responded: Judges 1, 
2, and 5 (as identified in Table 21) set the cutoff score'at 11; Judge 6 cet 
It at 10; and Judges 7 and 8 set it at 12. The median of these — and 
therefore the recommended cutoff score — is eleven correct answers to the 
twelve questions. This cutoff would have failed 192 of the pilot test 
examinees. A cutoff of 10 would- Have failed only 72; a cutoff of 12 would 
have failfd 52%. . . 
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Recommendations and Conclusions 



Eligibility requirements for exaninees 

Both the Working Committee and the Advisory Committee discussed the 
Issue of eligibility requirement s for ex aminees at their final meetings* 
Both committees endorsed formal education for dental assistants and dental 
hygienlsts and recommended that the Proficiency Examination be made available 
for testing of graduates of programs accredited by the American Dental Asso- 
elation^ Commission on Accreditation of Dental and Dental Auxiliary Educa- 
tional Programs. The successful completion of all modules by these examinees 
would indicate general proficiency in dismal ^issisting or, with the successful^ 
completion of a performance test in calculus removal, dental hygiene, as well 
as in the specific knowledge and skills tested In the examination* 

The Advisory Committee and the Working Committee also recognised that 
the Proficiency -Examination could be given to axaninees who arc not graduates 
of accredited programs in dental assisting or dental hygiene** The two 
commmlttees recommended that, for these examinees* the Proficiency Exaraination 
should be used only as an indication of proficiency In the knowledge and skills 
tested, and not as a complete evaluation of the examinee's general proficiency 
in dental assisting or dontal hygiene. 

Separate nodules of the examin a tion 

The Advisory Committee and the Working Committee discussed the possl- 
biA.lty that a credentlallng agency might want to use only a portion of this 
examination and the possibility that an individual examinee might want to be 



*For example, persons trained on the Job or in the military. 




credentlaled in some but not all competency areas* The committees reccamended 
that the individual modules of the examination could be used separately, 
provided that each examinee takes both the written and performance portions 
of each module he/siie takes. , 
Observers for the Perfornance Tests 

Training of observers. The observer's role in the performance tests 

J. 

ta critical. The performance test scores will be valid only if the observers 
have accurately recorded the examinees' actions during the tests. The 
observer's task is also a difficult one. The observer must know what 
specific actions to watch for, how to recognize then when they happen, and 
where to record thetn on the chart. The observer must be able to record each 
specified action quickly, so as not to miss the next. The observer's task 
Is tompiicatod by the fact that examinees do not all do the specified actions 
In the sane order. Therefore, the observers cannot be expected to do an 
adequate job unless they have hr.; the opportunity to practice using the 
perfornanrc checklists to record the performance of persons doing the tasks 
Included in cht- performance tests. In addition, each observer should have 
tJu' opportunity to discuss the interpretation and use of the checklists with 
other observers, after this first practice observation. Every observer 
should have these opportunities before serving as an observer of actual 

■J 

candidates for crt'dent la 1 Ing. 

How nany observers? One of tho greatest impediments to the use of 
performance tests is the cose of administering them. A major component of 
this cost is observer tine. One ohv>us ly to reduce the amount of observer 
time required for the performance tests In ti,e i'roflclency Examination is to 
use .inly ce observer per e v^inrc. Ncifher the Working Cocuyltte*. nor the 



Advisory Committee endorsed this solution* In some cases a compromise solu-* 
tlon might be possible— the use of a second observer for those checkpoints 
that describe aspects of the product of the task, rather than the process* 
This compromise solution could be used with the performance tests In placing 
and removing a periodontal dressing, preparing a study cast, taking an alginat 
Impression, polishing an amalgam restoration, taking v^ital signs, and coronal 
polishing* 

^ One examinee at a time for each observer is probably the practical 
limit* Observing one examinee at a time is difficult enough; observing 
two at a time and accurately recording their performance could result in 
confusion, Invalid scores, and inadequate protection of the patient (for 
those tests that use real patients) • One possible exception to this rule 
,l8 the performance test in dencal radiography, in which the observer does 
not observe the entire procedure* 
Administration of the performance tests 

A credentiallng agency planning to adminii^ter the performance tests 
to any substantial number of examinees will have to confront several problems 
of logistics: providing equipment and materials, training observers, 
arranging for patients, scheduling of the tests, set-up of the testing 
area, verification of candidates' identity, handling and scoi^ilng of the 
performance checklists^ and probably other problems as well* 'kilving these 
problems is beyond the scjcpe of the present project, but they will need to 
be resolved before the performance tests can be used op^arational ly* Further 
experient:e with the ^performance tests will tc necessary to find the most 
practical and efficient way^ of administering them to luCger numbers of 
examinees* 
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Additional test development 

If the Proficiency Examination is to become the basis for ar 
ongoing program for credentlallng dental auxiliaries, some additional 
test development work will be needed. 

Alternate forms of the written tests . Alternate forms are necessary 
to prevent the specific questions on the test from being known In advance 
by examinees. If examinees know the specific questions they will be asked , 
their answers to those questions'* will not be a valid sample of their knowledge 
of thie topics tested. Therefore, now that the development of a Proficiency 
Examination has. been shown to be practical, ETS recommends that one or more 
additional forms of the written tests be developed. 

AUernat^ forms o f 'the performance tests ! Most of the performance 
tests will not require alternate forms. Unlike the written tests, they 
each tost an entire task, not n «5ample of discrete points of knowledge. 
However, the performance tests in assisting with restorations, placing and 
removing a. periodontal dressing, and • polishing an amalgam restoration will 
require alternate forms to prevent examinees from specializing in the 
particular teeth or areas of the mouth specified for the tests. The 
development of alternate forms of these tests would be a brief, simple, 
and straightforward task, involving only minor modifications. 

Two other performance tests present more extensive problems In 
the development of alternate forms. These are the tests in oral hygiene 
instruction and charting from dictation. In each case, thd development 
of an alternate form would require the creation of a new "patient". For 
the test in oral hygiene Instruction, each new form of the ttst would 
require .ifcw intraoral photoa, dentil chait, medical and dental history, 

% 

M 

er|c 9 1 
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and Instructions to the patient actor, and possibly some changes in the 
performance checklist. For the test in charting from dictation, each new 
form of the test would require a new script for dictation, dictation tape, 
and set of multiple-choice questions. (However, these questions are 
easier to develop than most of those on the written tests*) In both'cases 
the amount of work involved would be substantial* 

A performance test in calculus removal ? The Proficiency Examination 
in its present form doQs not contain a performance test in calculus removal* 
The examination could be supplemented with the calculus removal portion of 
the ADHA Clinical Evaluation * It would also be possible to develop a 
performance test in calculus removal, in which. the examlnee^s technique 
is observed and recorded on a performance checklist* 

Continuing test developjient * ETS strongly recommends that if the 
Proficiency Examination becomes the basis for fn ongoing credential ing pro- 
gram, new forms of the written tests should be developed periodically* 
This continuing test development is necessary for two reasons: to prevent 
examinees from knowing the test questions in advance and to keep the tests 
current with advances in dental technology* The latter reason applies to 
the performance tests as well as the written tests* Therefore, ETS 
recommends that the entire examination be reviewed periodically by a 
committee of experts in dentistry, dental assisting, and dental hygiene 
and re^^lsed if necessary to keep it technologically up no date* 
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ADDENDA 



Teat Site Personnel 



Cabrl llo College^ Apt^s. California 



James L. Monahan, DDS, coordinator 
Joyce Herceg, RDH, observer 
Rosemary Baker, RDH, observer 
Gerry Hodges, RDH, observer 
•> Cherl Lusk, RDH, observer 
Betty Zachary, RDA, operator 
Barbara Sexton, RDH, operator 

M oines Area Comniunlty College. Ankeny. Iowa 

Sharon Moore, CDA, coordinator and observer 
Diane Schroeder, CDA, observer 
Leone Young, RDH, observer 
.Debbie Phannensteil , CDA, observer 
Alberta Lee, CDA, observer 
David Llewelyn, DDS, operator and observer 

\ 

,3 

Edmonds Community College > 'Xynvoodt Washlnf^ton 

Jdanne L. Svein, CDA, coordinator and observer 
vSandy Phillips, CDA^, observer 
Marsie Shorwood, -RDH, observer and operator 
Robert Lockyer, DDS, observer and operator 

Guilford Technical Institute^ Jamestown, North Carolina 

Charles Marcus Vaughn, DDS, coordinator and operate 

Henrietta Andrews, RDH, observer 

Patricia Hedrick, CDA, observer 

Ann Powell, RDH, observer 

Lillian Koleszar, RDH, observer 

Melissa Crowley, RDH, observer 

Loyola University^ Maywood, Illinois 

Marjorle Cletcher, CDA, coordinator 
Elizabeth Wunderlich, CDA, observer 
Sue Sauer, RDH, observer 
Andrew Bronny, DDS, operator 
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' Maricopa Technical Community College ^ Phoenix Arizona 

Mary Fllllnghan, CDA» coordinator and observer 
Karen Valetta, CDA,, observer 
Richard A« Flottmant DOS, operator 

/ 

Middlesex County College> Edl^^ont New Jersey 

Carolyn Breep, CDA, coordinator and observer 
-Harmon C* Zacune, DDS, observer 
Arthur Nagy, DDS, operator 

ShorfeUnc Community College, Seattle. Washington 

Barbara Renshav, RDH, coordinator and observer 
Susan Dougal, RDH, observer 
Carolyn Johnson, RDH, observer 
« Kathy Forbes Lovell, RDH, observer - r 

Southern Illinois University. Carbond e. Illinois 

Mary Edwards Callaghan, RDRNioordinator and observer 

Melitta T. Ebner, RDH, observer 

Josephine M. Leonard, RDH, observer 

Jo Ellen Wolaver, RDH, observer ' )f 

Gerald J. Mlckel, RDH., operator 

Unlversl^v^of Detroit, Detroit. Michigan 



Jdhi Newman, RDA, coordinator 
Jane Yorki RDH, observer 
Nancy Polcyn, RDA, observer 
Pam Deshong-Zarb, RDH, observer 
Jennifer Dye-Kazanowskl , CDA, observer 
Diane Gumbrecht, CDA, observer 
Dennis Corona, DDS, operator 
Gary Bolosa, operator 

University of New Mexico^ Albuquerque. New tlexlco 

Clara Mlera, CDA, coordinator and observer 
Demaris Wright, RDH| observer 
Victor Sandoval, DDS» operator 

♦ 

University of Texas. San Antonio. Texas 

Jack L. Hardage, DDS, coordinator 
Jane D, Carr, RDH, observer 
Christine A. Leitiione, CDA, observer 
Claude E. Padgett, PDS, operator 
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ETS Project Staff 



Many ETS staff raemb^jrs worked in the Dental Auxiliary Examination Project, 
and it seems appropriate to acknowledge their contributions. 

Samuel Bocchieri. assisted in the development of the'wrltten tests* Susan 
Carlson assisted in the development of the pJrformance tests. ' Amiel ^harolT, S 



Lee Schroeder, Andre Diaz, Jayme Zi/to-Wheeler, and Jacqueline Hiller did the / 
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Description of the Examination 

The Proficiency Examination for. Dental Auxiliaries consists of a 
written multiple-choice test and performance tests in five competency 
areas: 

I. Chairs ide assisting 
II. Laboratory procedures 
III. Diagnostic aids 
IV. Patient aducation 
V. Dental therapeutics 

The specific tasks involved in the performance tests in each com- 
petency area are these: ' . 

I. Chairside assisting 

Assisting with restorations 
Completing tray setups 

Preparing the unit and chair and positioning the patient 
Assisting with the preliminary examination of the patient 
Assisting with the adminstration of a local anesthetic 
Assisting with the placement and removal of a rubber dam 
Assisting with cavity preparation 

Assisting with an amalgam and ,a composite restoration 
Surgical scrub and glove 



II. Laboratory procedures 
Study cast 

* 

III. Diagnostic aids 

Taking of vital sigr 

Taking an alginate impression for a study cast 
Dental radiography 
Charting from dictation 



TV. j^atient education 

Instructing the patient in oral hygiene practices 



y. Dental therapeutics 

* Coronal polishing 

Polishing of an amalgam restoration 
Placing and removing a periodontal dressing 

Four of these tasks will be performed on actual patients: coronal^ 
polishing, dental radiogra^)hy , alginate impression, and takfng of vital^ 
signs. Manikins will be used for polishing of amalgam restorations and 
for all tasks in chairside assisting. The periodontal dressing will be 
placed on a person who does no^ actually have a gingival wound; this 
person may be a patient , or someone else who may be available. The per- 
formance test in patient education will be done with a patient-actor, or 
"programmed patient" - a person who is not actually a patient but has 
been trained to respond in a specified way to the examinee. The perfor- 
mance test in chairside assisting will require an operator who has been 
trained for this examination to follow a specified procedure for the tasks 
on the test. 
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All the performance tests (except for the test in charting from dic- 
tation) require an observer to record the examinee's performance on a 
performance checklist. The observer also is responsible for preparing 
the examining station and, in those tests done with actual patients, 
for protecting the patient. 

Early tryouts of this examination vrLll require two observers for ■ • 
each examinee, to deterr'.ne where Inconsistencies between observers are 
likely to occuiT. The two obscM vers should observe independently and 
mark their checklists separately, without communicating with each other 
in any way until they have finished marking their checklists. 

Personnel Required for the Performance Tests 

As the administrator, you will be responsible for recruiting and training 
the observers, the patient-actor, and the operator. You will also 
have to arrange for the proceaures that involve real patients to be 
done under supervision of a dentist. (The dcnt^lst need not actually 
observe these operations for each patient buL must be available to per- 
form a preliminary examination on each patient and to deal with any 
emergencies that may arise.) 

Observers . Tt.e number of observers you will ne^d depends on the 
number of examinees you intend to test at one-time, since each examinee 
must be observed by two observers. (Testing more than one examinee at 
a time will make the observers' job easier, allowing them to specialize.) 
The observers should be experienced den:al auxiliaries who are thoroughly 
familiar with the procedures they will be observing. If possible, each 
team of two observers should include one instructor and one practitioner. 
Every observer should be thoroughly familiar with, all the performance 
checklists he/she Ml be using and should practice using them at least 
once before observing any actual examinees. In addition, each observer 
should be given a write-up of the emergency procedures used at your 
institution. 

Operator. The operator for the test in chairside assisting need 
not be a dentist, but must be a person who is skilled in the uso of 
the handpiece. (A dental student or an expanded-f unction dental auxiliary 
might be a good operator for the tost.) The operator must be familiar 
with the procedure for ^he test and should practice it at least once, 
preferably with the observers, before serving as operator for any actual 
examinee. " 

r , 

Patient-actor . Tlie patient-actor for the test in patient education •• 
must be trained to play the role of the patient in a way that is ^ 
realistic and consistent with the specifications contained in the patient s 
"program." This person need not have any professional knowledge or 
background in any dental occupation, but must be able to learn the 
"program" thoroughly and Improvise realistically in case the examinee 
asks a question not covered in the program. If possible, the patient 
actor should be someone not known to the examinee in real life. 
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Dentist on alte > E^ch test site must have a dentist present 
during the performance tests that are to be done on actual patients. 
The dentist should examine each patient for any contraindications 
to the procedures to be done. If there are any contraindications to 
any of the procedures, those procedures should not be done on that 
patient. The dentist should also be available to deal with any 
emergencies that may occur during the performance tests that involve 
a ctu al p a tient s. — 



The dentist may also serve as one of the observers or as the 
operator for thie test In assisting with restorations, 1J[ he/she has 
been trained as an observer or operator for the test. 

Patients for the Performance Tests 

The performance tests in taking of vital signs, taking an alginate 
impression, coronal polishing, and dental radiography require actual 
patients. You may require each examinee to provide a/patlent, or you may 
arrange for your institution to provide patients. It is not necessary 
to have an examinee do all the above tests on the same patient, although 
it may be convenient to do so. 

The patient for the coronal polishing task must have extrinsic 
stain but must have all calculus removed before sitting as a patient 
fbr the polishing task. The patient for the dental radiography task 
must be a person for whom an x-ray series is needed for a valid diagnostic 
purpose. 

The task of placing a periodontal dressing does not require an 
actual patient but does require a person who does not have an actual 
gingival wound. This person may be a dental patient or any other 
person who is available, except another examinee. 

Ins tractions, f Qr_ .thjg. ..Examinees _ .. . . .... ._. . 

Kach examinee should receive a copy of the Advance I nstructions to 
the Examinee at least two days before the examinee takes the tests. In 
addition, the ' observers will have On-site Instructions f or the Examinee 
for each task in the performance tests. The examinee should not. receive 
tlie on-site instructions for any task until 15 minutes before he/she is 
to bej^in that task. 

Ad£in is tc-r in ^ Ll \ o.^ Perfo rmance T ests 

The performance tests arc intended to be administered separately 
from the Witten test^' with the exception of the performance test in 
charting from dictation. ' This test is to be administered along with the 
written test, ns described below. 

The order in which the performance tests are ^iven is not important, 
except tliat the alginate impression must be made before the study cast. 
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You may test more than one examinee at a time, if you find it 
convenient. However, each pailr of observers can observe only one 
examinee at a time. Therefore, if you test two examiijees at the same 
time, you will need four observers, and so on. If you decide to test 
more than one examinee at a time, you will wai.t to avoid having them 
work on the same tc.sks at the same time, since some tasks require 
- special p e r s onnel- ^ tho ope ra^;-the- p a t ien r -ar. fo r ) . — Al_so^_.if_y,ou 



test more than ono examinee at a time, be sure to station the examinees 
in the operator ieu in such a way that they cannot watch each other 
working. Do not let anyr examinee watch another examinee ta king any of 
performance testa . 

Performance Test in Charting from Diciation 

The performance test in charting from dictation is to be admin- 
istered along with the written test, usiijg the following procedure. 

1. Give each examinee two red pencils, two blue nencils, 
a copy of the charting answer sheet (with the 

" anatomical "chart at the top), and-a copy of the examinee^s 

/ instructions for the performance test in charting from 
dictation. 

2. Read the examinee's Instructions aloud to the group. 

3. Play the dictation tape cassette. Make sure all examinees can 
hear the tape clearly. 

A. Administer the written test. Make sure the examinees do not 
• . - hand in their charts along with their written test materials. 

5. Give each examinee a copy of the test questions for the perfor- 
fliance-test ln_chartlng» .which the yL.are to answer on the_ basis^ _ 

of their completed charts. 

6. Collect the answer sheets and test booklets for the performance 
test in charting. « 

You will be provided with two copies of the dictation tape cassette 
and also a copy o£ the script from which the tape was made. 
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Materlals Needed for the Performance Tests 
!• Chalrslde Assisting 

Assisting with Restorations 
Assisting cart 





Pretaeasured amalgam capsules 
Dappen dish or amalgam well 

Sqx^fteze cloth ^ 
Calcium hydroxide base material 

Composite restorative material: original containers containing 

only a iSmall amount of base and catalysw 
Mixing pads for calcium hydroxide base and composite material 
Spatulas for mixing calcium hydroxide base and composite material 
Topical anesthetic 
Varnish 

Alcohol-saturated gauze 

Manufacturer's instructions for mixing amalgam 
Manufacturer's instructions for mixing composite material 
Manufacturer's instructions for mixing calcium hydroxide base 
Triturator (amalgamator) , set to proper speed for type of 

amalgam to be used, with proper mixing time posted 
Lubricant for rubber dam 
Bib and chain 

Incomplete tray setup for amalgam restoration 

Mat or rack for hand instruments 

Saliva ejector 

Wedges 

Gauze squares 

Cot ton- tip applicators 

Cotton rolls 

Cotton pellets: large and small 
Articulating paper 
Dental floss 

Rubber dam, stamped but not punched 
Rubber dam napkin 



Incomplete tray setup for composite restoration 
Mat or rack for hand instruments 

Polishing and finishing discs: coarse, medium, and fine 
rinishing strips: fine grit 
Plastic matrix strips 
Wedges 

Gauze squares- 
Cotton-tip applicators 
Cotton rolls 

Cotton pellets: large .and small 
Articulating paper 
Dental floss 
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Instruments for completing tray setups 



Hand Instruments: 

Amalgam carrier, double-ended 
Amalgam condenser, double-ended 
Gingival margin trimmer, double-ended 

Cleoid-discoid carver 

Wedelstaedt -ehisei^ — ■ " 
Hollenback carver) 
Jacquette scaler^Sdouble-ended 
Spoon excavator, double-ended 
Wax spatula, double-ended 
Explorers - two (2) each 
Periodontal probe 
Small T-burnisher 
Mirrors - two (2) each 
Plastic instrument (metal) 

Composite filling instrument (double-edged, nylon) 
Calcium hydroxide placement instrument 
Cotton pliers, non-locking 

Gold knife £r scalpel with #12 blade mounted 

Friction-grip burs of the following types: 
Round 

I 

Inverted cone 

Straight plaincut fissure • 
Tapered plaincut fissure 
Straight crosscut fissure 

Tapered crosscut fissure ^ 

Pear-shaped 

Flame-shaped 

End-cutting 

A composite (12-fluted) finishing bur, a diamond bur, and an 
abrasive stone, all friction grip and all in the same shape: 
pear-shaped, flame-shaped, or round 

Latch-type burs of the following types: round cutting, inverted 
cone, and round finishing 

Moores' Mandrel, for latch-type contra-angle 

Straight mandrel 

Latch-type mandrel with screw head 
Bur block 

Slow-speed straight handpiece 

Contra-angle (latch- type) 

Bur removal tool 

High-speed suction tip 

Rubber dam punch 

Rubber dam frame. Youngs 

Scissors 

Rubber dam clamp forceps 

Rubber dam clamps for anteriors, bicuspids, and molars - 3 of e 
Tofflemire matrix retainer with bands: //l,'//2, #13, //14 - 3 of 
'Aspirating syringe 

Carpules of local anesthetic in their original containers 
Needles for syringe 
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Surgical Scrub and Glove 

Liquid soap or soap in sterile package 
Towels \ 

Packaged sterile surgical gloves (In full range- of sizes) 
Fing e r - nail c l ea ner — — 



Handbrush 

II . Laboratory Procedures; Studv Cast » . 

Dental stone 

Manufacturer's instructions for dental stone 
Spatula 

Rubber mixing bowl 

Scoop for dental stone 

4x4 tile or glass slab, or wax paper 

Vibrator 

Lab -knife..- 

Model trimmer 



III. Diagnostic Aids 

Taking of Vital Signs 

Stethoscope- 
Blood pressure cuff 
Clock or watch with second hand , 
Pen or pencil 

Taking an Alginate Impression for a Study Cast 

Alginate impression material - regular and fast set 
Manufacturer's instructions for alginate material 
Two bowls and spatulas for mixing alginate material 

Scoop 

Water measure 
Impression trays 

Labels or tape for labeling impression trays 
Pencil or pen, disinfected with alcohol 
Cloth towels 
Dental floss 

Wax for bile registration 

Lab kniie (for trimming wax) , ^ 

Water bath ^ 
Utility wax (rope wax, periphery wax) 
Wax strips (fj^ extending tray) 
Mouthwash 

Small ruler o.r gauge marked in millimeters 
Mirror 

Saliva ejector 
tmesis "basin 
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Dental Radiography 

X-ray film: double film packs and jingle film packs 
Bite wing tabs , 
Bite block film holders 

-Fnm"nib"urts'T6r I4-film periapical' series, irlus bite wings 

Lead apron 
Headrest covers , 
Alcohol-saturated gauze 
Dental ^ mirror 

Patient Education . ^ ^ 

Hand mirror 

Toothbrushes: both hard and soft, of diff erent • colors (for ease of 

identification by .observers) 
Dental floss 
Dental manikin 
Pencils 

Paper * 3 



Dental Therapeutics 
Coronal Polishing 

A slow-speed handpiece that has been properly cleaned, lubricated, 

and teste d 
Prophy angle 

Several rubber polif^hing cups 

All polishing cups must be properly sterilized. 
Two bristle polishing brushes, properly sterilized 
Mirror 
Explorer. 
Cotton rolls 
Dental floss 
Gauze 

Vacuum tip 
.Saliva ejector 
Cotton-tip vapplicators 
Dishes for polishing agents 
Dish for soaking brush 
•Polishing agents - coarse, -medium, and fine; labeled 
Tin oxide 

Ring holder for polishing agents 

Alcohol 

Dental tape 

Towel 

Towel clips 

Spatula (for putting polishing agents into cups) 

Waste disposal sack 

Disclosing solution 

Dish lor disclosing solution 

Disposable cup 

105 



-9- 

« f 

Po lishing of an AftUlRam Restoration 

Finishing burs (latch type) : *. ^ 

Round: //2, H 

Flame-shaped: #2,^ U " ^ . 

Pear-^shaped : #2, ifj^ 
Finishing 'Stones (latch type): round, flame-shaped, and pear-^shaped. 
Impregnated rubber points 
Mandrel: latch type 

Mounted bristle brush, soft, for straight handpiece ^ 
Finishing discs: medium and fine 
Finishing strips: medium and fine 

Mirror . • ^ ^ 

Explorer 

Cotton rolls 

Saliva ejector ^ 
Dental ^IcTss 
Dental tape 

Polishing agents: flour pumice and tin oxide 
Cups for polishing agents 



Placing and Removing a PeriodonLal Dressing 

Periodontal dressing material (non-eugenol) 

Surgical powder 

Mirror 

Explorer 

Probe 

Plastic instrument 
Curette 

Cotton pliers * 
Mixing pad 
Tongue blade 
Gauze 

Petroleum jelly 

Saliva ejector 

Color transfer applicators 

Pen or pencil, disinfected with alcohol 

Cotton rolls 
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Charting; from Dictation 



Cassette player 

Red and blue pencils - two of each color for each examinee 



* 
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Description of the Examtnatlon 

# 

The Proficiency Examination for Dental Auxili^iries consists of 
a written multiple-choi^ie test and performance tests in five ^ 
competency areas: 

I. Chairside assisting 4^ ^ 

\^ II* Laboratory procedures 

III. Diagnostic aids 

IV. Patient education 

V. Dental therapeuticSo 

The specific tasks involved in the performance tests in each 
competency area are these: 

I. Chairside assisting 

Assisting with restorations 
Completing tray setups 

Preparing the unit and chair and positioning the patient • 
Assisting with the preliminary examination of the patient 
Assisting with the administration of a local anesthe^Cic 
Assisting with the placement and removal a rubber dam 
Assisting with cavity preparation 

Assisting with an amalgam artd a composite restoration 
Surgical scrub and glove 

II, Laboratory procedures 
Study cast 

« 

ill. Diaono^tic aids * . ^ . 

Taking of vital sij^ns 

Taking an alginate impression for a study cast 
Dental radiography 
Charting from dictation- 

iV. Patient education ^ 
Instructing the patient in oral hygiene 
practices ^ 

V. Dental therapeutics 

Coronal polishing 

Polishing of an amalgam restoration 
Placing and removing a periodontal dressing 

o 

Your performance on each of these tests will be recorded by 
two observers. One of the observers will direct the procedure in 
those tests that require it. # ^ 

J.07 



T he P e rformance Tests 
Chalrslde Assisting • • 

Assisting wltK Restorations 

Your performance test in agisting with restorations will consist 
of .,the following tasks: 

• Select instruments to complete f)artially prepared tray setups 

• Prepare the linit and chair and position the patient 

• Assist with;the *prelimin^ry examination of the patient 

• Assist with the administration of a local anesthetic 
Assist with the placement and removal of a rubber dam 

• Assist with cavity preparation 

• Assist'with an amalgam restoration and a composite restoration 

Ihe procedures that require a patient will be done an a manikin, 
rather than "on an actual patient. Nevertheless, you are to treat the 
manikin as if it wer.e a live patient and take all prec autions th?t you 
wo uld take with a li\^e patient . You will receive a copy of the patient 
clinical record. Read it carefully before you begin the problem. It 
will tell you what specific procedures are to be done". 

<• * ^ ■ 

To save time, the operator will not actu-nlly complete alJ. of the 
procedures on the test, ^Nevertheless, you are to assist v^lth the pro-- 
cedures in the same way that you would with a real patient until the 
observer tells you to do ocherwise. , I 

I" 

The operator for this' test will work right-handed fr^^n a sitti../ 
position. All Instrument exchanged will take .place within a tran^.fer 
zone outlined by the rubber cam ^^ame. Use the, same assisting pro- 
cedures thgt you would use in as. ..sting an o-peratoi-^ith whom you have 
never worked before. " , 

■ 

The assisting cart will be prepared for you. The tray setups will 
be partially prepared in advance. You will compi<i*e the 6^;tups by 
selecting and placing instruments. (You'wiU receive inspruetionr. 
telling you h6w to complete the setups.) 

The equipment used in the tost will include non-loc]<inK cottcm 
pliers, an aspirating syringe, a latch-type contrri-an^le, and a Younj-s 
rubber dam frame. 

Manufacturer's instructions: will be available fo!" you lo us<; in 
preparing' materials. You will mix the a;7Kilgain from preweasured capsuK- 
using a triturator. ' " ?>■ 

The rubber dam will be placed' by the following procedure:^ 

1. Assistant punches holes ip rubber dam. ^ ' 

2. Operator places rubber dam clamp on anchoi- tooth. 

3. Operator places rubber dam. 

A. kssl0f^t places rubber dam napkin. • 

5h OpeW^r stretches rubber dam over anchor tooth and clamp 
and over tooth farthest away from anchor tooth. 

6. Assistant Urates anchor tooth with floss to hoJU dam in place 

7. Assistant places rubber dam oVi frame. 

8. Operator passes dam between patient's contacts. ^ • . 

9. Operator places an additional rubber dam cldmp. 

10. Operator inverts rubber dam. » • * 
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Surgical Scrub and Glove 1^ 

This test requires you to perform a Burgicv^l scrub and glove. 



Laboratory Procedures; Study Cast 

For this test you will pour and trim a study cast from an alginate 
impression, using the following procedure: 

1. Pour the cast in dental stone, using a single-pour technique, 

2. About three minutes after pouring the cast, perform a "bench 
trim," using the lab knife to cut away most of the excess 
material before it hardens, 

3. About one hour after pouring the cast, finish the trimming 
of the cast using the model trimmer. 

(Between steps 2 and 3 you will be taking other performance tests.) 



Diagnostic Aids 
Vital Signs 

For this tost you will take the patient's pulse, note the patient's 
respirations, and take the patient's blood pressure, using a stethoscope 
and blood pressure cuff. 

>• ♦ . 

Taking an AU;inate Impression for a Study Cast 

For this test you will make an alginate impression' for a study 
cast on a live patient. - You will take impressions^5¥*WC^i upper and 
lower arches. You will a^so take a wax bite. You are to assume 
that the patient has never had an alginate impression taken before . 

pe ntal Radiography 

Fur this test you will exposp and mount a full mouth series 
consisting of 14 periapical films and bite wings. You may use 
either bisecting of paralleling technique. Bite blocks will be pro-. 
v?ded, but you "^may .bring anot.her type of film holder and use it instead. 
You will" havei'30 minutes to make the exposures. 
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Chartlng from Dictation 

This performance test in charting from dictation does not 
require you to use a particular charting system • You may use what- 
ever system you are familiar with. You are not being tested in the 
use of a particular formal system, but on your effectiveness at 
recording information about a patient's dental condition and inter- 
preting what you have recorded. The test will be given in two parts; 

1. You will receive red and blue pencils to use In 
charting. You will also receive a sheet containing^blank 
charts of two types: anatomical and schematic. Your may 
use whichever chart you prefer. You will listen to a tape 
recording of a dentist dictating information to be charted. 
While you listen to the tape recording, you will record the 
information on your chart. 

2. You will be given a series of questions based on the 
information contained in' the tape recording. Using your 
chart, you will answer these questions from the information 
you have recorded on th^ chart* 

You willT^e taking other tests in between parts 1 aad 2 of thTs 

test. Make sure to keep your chart until you have completed parte 2 
o^f tl?is test. • • 

Patient Education • ^ , 

This test is a tect of your ability to conduct a patient edu- 
cation session that -will improve the patient's oral hygiene practices. 
The patient's dental chart and history form will be provided. Your 
task is to identify the patient's dental problems that can He helped 
by home care and to instruct the patient in the necessary oral 
hygiene procedures."" It will he up tJ you to decide what to teach 
the patient and how to teach it. Pencils, paper, tO( tthbrushes , 
dental floss, a hand mirror, and a dental manikin will be provided. 
You will have 20 minutes for this test. 



* Dental Therapeutlcs N 

- Coronal Polishing ' . 

This test will be done on a live patient. Any calculus ^will have 
been removed prior to the test. You Will polish the patient's teeth 
and remove any extrinsic stain. You will be evaluated on the polishing 
of two quadrants. The observer will tell you which two quadrants to 
polish for evaluation. You will not be permitted to use a disclosing 
agent. You will have 20 minutes to complete the polishing. 

' Polishing an Amalgam Restoration 

For this test you will polish an amalgam restoration in a manikin. 
You will have 10 rai;iutes to polish the restoration. 
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Placlng and Removing a Periodontal Dressing 

This test will be given in tvo parts. In the first part you 
will place a periodontal dressing and instruct the patient in 
postoperative care. In the second part you will remove the dressing 
and instruct the patient in postoperative care. The two parts will 
be separated by 30 minutes or more, during which time you will be 
taking other performance tests. You will have 15 minutes to mix, 
place, and ^rim the dressing and 5 minutes to rarnove i';. 
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Pilot Test 
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Proficiency Examination for Dental Auxiliaries 
General Instructions for Performance Test Observers 



Description of the examination 

The Proficiency Examination "for Dental Auxiliaries consists of ' 
written tests and performance ^ests in five general areas or dental 
auxiliary work: chairside assisting, laboratory procedures, diagnostic 
aids, patient education, and dental therapeutics. The purpose of the 
performance tests is to test what the written tests cannot adequately 
test: the examinee's skills and procedures in actually performing a 
selection of important tasks.* 

The specific tasks involved in t)ie performance tests la ^ach 
competency area are these: 

I. Chairside assisting 

Assisting with restorations 
Completing tray setups 

Preparing the unit and chair and positioning the patient 
Assisting with the preliminary examination of the patient 

. f^^ ^i^rin^ w i th the a dmi nistratio n of a local anesthetic 

Assisting with the placement and removal of a rubber dam 
Assisting with cavity preparation 

Assisting with an amalgam and a composite restoration 
. Surgical scrub a:;d glove 



11. Laboratory procedures • 

• * Study cast ^ . . • 

III. Diagnostic aids 

Taking of vital sign^ 

Taking an alginate impression for a study cast 
Dental radiography ^ 
Charting from dictation 

IV . Patient educn t j on 

Instructing the patient in oral hygiene 
practices 



V. Dental therapeutics 

Coronal polishing 

Polishing of an amalgam restoration . 
Placing and removing a periodontal dressing 
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General Instruptions 

Observers 
* 



Pour of these tas.s will performed on ual p t n^^^^ oronal 
polishing, dental "biography, algina e Impression and t^^^^ 
signs. Manikins will used or polishing of am^^^^^ ^^^^ 
for -all tasks in chairside f ? * ^^^.^'f ° gingival wound; this 

placed on a person who does .not ^^^f^^^^.^^^^/^ available . • The 
'person may be a Pf -,^~tioi will b^Ln: with a patient-actor, 
per.formance test in patient education wi ^ ^ patient but has 

Ir "programed patient' The perfor- 

been trailed to respond in a specified to % operator who has been 

nuance test in chairside a-is ing_ will -^^f/,,^,^ ^^.^.^dure for the 
trained for this examination to follow a specxrie i ^ 



tasks on the test. 



The perform^rtee tests have been constructed to test both the 
ine periomwiiv results the examinee obtains, 

procedure the examinee uses and the ^^^^^'J , important 

consistently good results. . ^ 



Admlnlster ln p the per formance tests 



Fairness r-qulres that all examinees take the test under conditions, 
that Ire's nearly the -ama as possible. As the 
^^^bl^forieEarln^^M 

^.::r ^r r.srrLfsfii:?^^ 

-df .11 ^^:ir:r"yo^rrrnoriU":^ 7enraraS.lll,ry practice and 
instruction. 

ror.each task on the . .rfor^ance tests. ^^flZZ' IZ^'l^Zi: 
S^e^:-r§^^^HI^tfl^f^- i d coUect the. .s 

•■ oon as t""LJlnee has Ilnlshed the task :0o not let the exa^.nee 
keep the on-site Instructions for any o£ the tasks. 

, Ob^^^rylns^tts^ej^^^ the ex.^lnee's 

., — ur ..m .o. .s an ohserver is - ^^X^^/^Bi^^^ 

actions in performing I, J" fnLber of specific things the 

7-^rj!^^ojMASl' ""'=\^i''L examinee, you will check spaces on 
,.^e shouW do. AS you^wa.ch tl e fJlUlJ^^,^^, ,he examinee d.d- 
tho checklist marked, -es qr no to i ^^^^ ^^^^ be. 

or dlti not do .ach specific 'hing. In J„„ .^e ci-ecklist.) 

asked to record other types «P^^i"^/"„°S^%he things on ' the check- 
In general, the exan.iree Is £ot "^"^"^ Checklist. When It Is 
list m the order m which ^^ey a, Pea on h ^^^^^^^ ^ 

rckfis't wT/i z 

t^Z^i:^ ^^n:tfar : :uauy doin. It. tell the 
exaltnee to do the things he/she is describing. 
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r^ifu t° effectively you will need to be thoroughly familiar 

with each performance checklis-t you will be using, so that you wUl 
know what to watch for and where to record what you have obse^vld Th. 
ca%%%u?^ '° ^/T' ^ performance checklist L to re^d it 

ta'k win/f P.^^^^^f "^^"R by having someone ^tuallj do tL 

ta.k while you observe and record his/her performance. You should also 
be familiar with the instructions to the examinee and with any other 
printed materials used in the test. ^ 

You should do your best to avoid giving the examinee any "feedback" 
."es^Jre or'fL-T"" '"^f"^ ''''' ^° indicate'- by word 

or^^";;\L^:p io^^r^ifc^^LL'^:^^^ ^'^T ^^'"^^ 

to prevent a risk of harm to ^he patflnt.r" ""^^ '° examinee, 

of this test there will be two observers 
watching at the same time. In these cases it is important that the two- 
Observers complete their checklists .independently , 'if you are working 
with another observer, do not look at the other observer's checklist 
w?H r-r" "H'Pleted yours. Do not exchange inforiAation or opinions 

onrl I f^^her observer until you have finished marking" your checklist. 

other-oEp^s checklist or discussed thT^mini^-^rf oiWc^T^^ 

nor change any marks you have made. 



For some experitpental administrations -of t^ese tests you will also 
be ^skud to make^a judgment of the examinee's perform^ce of each task 
as being adequate or Inadequate. TLese judgments ai;e not part of the 
test but will bo used to provide interpretive data to test users. 
st'^nd^rJ ^hat.you are not being asked to apply the highest professional 
standards of proficiency - only to judge whether the examinee's per- 
tormance would be adequate in a dental office or clinic* 

Several of the performance tests contain checkpoints statins, 
hxaminee uses aseptic technique . ..." These checkpoints refer to ^ the' • 
prevention of cross-contamination. However, if cross-contamination occurs 
because the examinee h.-^ neglected to disinfect a surface and this omission 
liaij already been noted on the checklist, do'nct count the same -error 
againsT:' the examinee in evaluating aseptic tecUnique. 
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As an observer, are responsible for pr otection of the patient 

during those tasks to be done on actual patientZ The obs^nTir*^ 

instructions include procedures for safeguarding the patient. Some of 
these procedures are a J so printed on the performance checklists as a 
reminuer to you. i)o _nj.x.lcit.. Hje _exnjni^ ,haL 
en^da nKers the paL _u>nt.- Tf the examinee ' siarL-; "^^^dTTometh ing~that 
endangers the patient, stop the ex.iminee, correct the error, mark 
your checklist, and then tell, the examinee to proceed. 

114 , 



Pilot Test 



Perforaance Test in Assisting with Restorations 



Instructions to the Operator 



You will be serving as the operator in a clinical performance 
test in chairside assisting. The "patient" for this test will be a 
manikin. However, tho examinee is to treat the manikin as if it were 
a real patient and to observe all precautions that would be appropriate 
with a live patient. You should do the same, except for the actual v 
placement of restorations (as explained below.) The examinee s tasks 
will include assisting with cavity preparations. 



Thc:se preparations may be partly completed advance, to save testing 
ti-c They may be preparations that were completed and partially 
restored during the testing of a previous examinee, as long as they 
contain enough excess material to test the exa.Tiinee s ability to use 
iiuc^ou to remove debris from the work area. 

■ Part of this test involves assisting with an amalgam restoration 
and a composite restoration. Since the purpose of the exercise is to 
test the assistant's skills, it is not necessary to finish these res- 
torationn. To sa^ time^e ope-rator should place only one increment 
oL amalgam and should not wait for the compos ire mairerial to- cu^e. 



Part of this test involves assisting with the placement and 
removal of a rubber dam. It may ^be necessary to lubricate the patient s 
teoch in order to place the dam. If the dam tears, disregard the tear 
.and proceed, unless'the tear is large enough to interfere with the 
cavity preparation or restoration process.. i • 

I ' 

Procedures 

In a test of ,]:his type, it is important that^all operators fbllow 
Lhc same sequence of operations. The operator will work right-handed, 
from a sitting position. Bur changes will be made by the assistant, 
who will retrieve the handpiece, change burs, and deliver the handpiece. 

Ideally, the operator should maintain fulcrums on the "patient" 
during the transfer of instr(iments. However, some operators may find 
this procedure difficult. In any case, all instrument exch anges for 
this test must he made within a transfer ^one outlined by the rubber dam 

— — ■ ' ' ■ ^ — ' ' 

frame . 
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Assisting with Restorations 
Operator 



1> 



The assistant's performance in this test will be recorded by observers 
using a performance checklist. The observers will check the examinee s 
performance of each instrument exchange and preparation and delivery 
of materials. Each tljne instruments are exchanged, the observers musu 
observe the exchange, then mark their checklists. The procedure has 
been standardized, so that all assistants taking this test will be 
tested on the same sequence of Instrument exchd^^es. For these 
reasons, the entire procedure will be directed by one of the observers , 
who wili_tell the examine which instruments to deliver to you. 

Some types of . errors by the assistant arc difficult for anyone 
but the operator to detect. Patt of your job is to make these 
errors observable: 

1. If the assistant delivers the wrong instrument, say 



*'No, this is a 



I need a 



2. 



(The same applies to burs, finishing discs, etc.) 

If the assistant delivers the instrument to you in an 
incorrect or awkward position, look at the observers 
to make sure they are watching and shift the instrument 
to the correct position. The same applies to the 
syringe, the handpiece, etc. If yqu need to turn 
the barrel or the needle on the syringe, make sure 
that the observers can see you do it . 



Each time the examinee delivers an instrument to you, simulate 
working with it while the observers are marking their checklists. 
Continue to simulate working with the instrument until the observer's 
next command to the examinee. Because the observers will be directing 
the procedure, you do not need to remember the exact sequence of 
instrument transfers. You need only check to see that the examinee 
delivered the instrument or material specified by the observer. 

You should avoid giving the examinee any additional information 
not given by the observer directing the procedure. (For example, 
do not tell the examinee what size holes to punch in the rubber dam.) 

Before you enter the operatory, the examinee will prepare tray 
setups, prepare the unit and chair, and seat the "patient** • 



er|c - 
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Assisting with Restorations 
Operator 



Rubber dam procedure 

The procedure for placing the rubber dam will be as follows: 

1. Assistant punches holes in dam. 

2. Operator places clamp on tooth. 

3. Operator places rubber dam. 

A. Assistant places rubber dam napkin.. 

5. Operator stretches dam over clamp and over 
tooth. 

6. Assistant ligates tooth with floss. 

7. Assistant places rubber dam on frame. 

8. Operator passes dam between patient's contacts. 

9. Operator Inverts rubber dam. 

It may be necessary to lubricate the manikin's teeth for Installa- 
tion of the rubber dam. If the rubber dam tears, the operator should 
continue with the procedure unless the tear is so large as to interfere. 



The procedure for removing the rubber dam will be as follows: 

1. Operator removes rubber dam clamp. 

2. Operator cuts rubber dam with scissors. 
Qp-firatQT xe^igoyes, rubber dam and f ranie. 

Operator checks rubber dam for missingT ragments . 
5. " Operator examines patient with mirror and explorer. 
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